2016-01-14 15:47:58,701 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2016-01-14 15:47:58,730 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.SortedValuesPlatformTest, with platform: hadoop2-tez
2016-01-14 15:47:58,774 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-01-14 15:47:59,154 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-01-14 15:47:59,515 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides:
2016-01-14 15:47:59,729 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:47:59,731 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:47:59,852 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: 1D354BB1A9E5435C8B8ABC52C8A254BC
2016-01-14 15:48:00,087 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.117
2016-01-14 15:48:00,135 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.166
2016-01-14 15:48:00,137 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:48:00,138 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:48:00,140 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:00,294 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 15:48:00,465 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt
2016-01-14 15:48:00,507 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed
2016-01-14 15:48:00,781 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.4
2016-01-14 15:48:00,782 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] starting
2016-01-14 15:48:00,783 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:00,783 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:00,784 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"]
2016-01-14 15:48:00,784 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] parallel execution of steps is enabled: false
2016-01-14 15:48:00,785 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executing total steps: 1
2016-01-14 15:48:00,785 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] allocating management threads: 1
2016-01-14 15:48:00,786 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] starting step: (1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:00,786 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [apache*ip] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:00,802 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tez session mode enabled: true
2016-01-14 15:48:00,976 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:00,985 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:00,986 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:01,032 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001 doesn't exist and is created
2016-01-14 15:48:01,093 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:01,094 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:01,095 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001
2016-01-14 15:48:01,127 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786480985_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:01,169 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:01,186 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:01,194 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-01-14 15:48:01,194 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-01-14 15:48:01,194 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:01,233 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:01,238 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:01,243 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001/localmode-log-dir
2016-01-14 15:48:01,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001/localmode-log-dir/history.txt.appattempt_1452786480985_0001_000000, maxErrors=10
2016-01-14 15:48:01,244 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:01,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786480985_0001_000000, appSubmitTime=1452786481110, launchTime=1452786481125
2016-01-14 15:48:01,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:01,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:01,263 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:01,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:01,295 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 15:48:01,295 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 15:48:01,295 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:01,328 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:47536
2016-01-14 15:48:01,368 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:01,395 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 15:48:01,395 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 15:48:01,395 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:01,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:01,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786480985_0001_000000, startTime=1452786481435
2016-01-14 15:48:01,436 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:01,496 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:01,514 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:01,514 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1452786480985_0001, dagName=[BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:01,714 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:01,811 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786480985_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001/localmode-log-dir/dag_1452786480985_0001_1.dot
2016-01-14 15:48:01,815 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001/localmode-log-dir/dag_1452786480985_0001_1-tez-dag.pb.txt
2016-01-14 15:48:01,921 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786480985_0001_1
2016-01-14 15:48:01,921 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:01,922 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:01 Running Dag: dag_1452786480985_0001_1
2016-01-14 15:48:01,976 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:01,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786480985_0001_1, submitTime=1452786481714
2016-01-14 15:48:02,008 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,016 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:02,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786480985_0001_1, initTime=1452786481981
2016-01-14 15:48:02,020 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786480985_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:02,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:02,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:02,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_STARTED]: dagID=dag_1452786480985_0001_1, startTime=1452786482023
2016-01-14 15:48:02,024 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:02,026 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786480985_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:02,026 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C0B493A9663346E5ADB005D506F37D96 : {B4B66DEC01D946AAAA58980743758C63={InputName=B4B66DEC01D946AAAA58980743758C63}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:02,026 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B4B66DEC01D946AAAA58980743758C63, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:02,027 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,032 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,032 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,034 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]. Starting root input initializers: 1
2016-01-14 15:48:02,035 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1452786480985_0001, dagName=[BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:02,039 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B4B66DEC01D946AAAA58980743758C63 on vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,049 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:02,049 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AE9CB00F172D424DAA1215E1A113C882 : {A79F425B275C462BBC16EF900023880C={InputName=A79F425B275C462BBC16EF900023880C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:02,050 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A79F425B275C462BBC16EF900023880C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:02,050 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,050 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,051 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,051 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]. Starting root input initializers: 1
2016-01-14 15:48:02,052 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:02,055 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,060 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:02,063 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B4B66DEC01D946AAAA58980743758C63 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:02,063 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A79F425B275C462BBC16EF900023880C on vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,069 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] submitted tez dag to app master: application_1452786480985_0001, with dag id: dag_1452786480985_0001_1
2016-01-14 15:48:02,066 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:02,074 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:02,110 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:02,111 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,126 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:02,133 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A79F425B275C462BBC16EF900023880C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:02,138 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:02,139 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:02,147 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,149 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,150 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=30089AFED8C24CFF9C76F292A1703C4B, vertexId=vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:02,154 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=30089AFED8C24CFF9C76F292A1703C4B, vertexId=vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,157 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 410
2016-01-14 15:48:02,160 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B4B66DEC01D946AAAA58980743758C63 on vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,162 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 15:48:02,162 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A79F425B275C462BBC16EF900023880C on vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,168 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=30089AFED8C24CFF9C76F292A1703C4B, vertexName=C93856C0D38D44EA93187859B133FBD7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:02,175 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=30089AFED8C24CFF9C76F292A1703C4B, vertexId=vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,180 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:02,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C93856C0D38D44EA93187859B133FBD7, vertexId=vertex_1452786480985_0001_1_02, initRequestedTime=1452786482052, initedTime=1452786482177, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:02,181 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:02,184 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:02,184 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B4B66DEC01D946AAAA58980743758C63=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:02,185 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] parallelism set to 1
2016-01-14 15:48:02,185 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:02,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786480985_0001_1_00, eventCount=1
2016-01-14 15:48:02,207 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:02,207 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C0B493A9663346E5ADB005D506F37D96, vertexId=vertex_1452786480985_0001_1_00, initRequestedTime=1452786482026, initedTime=1452786482206, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:02,207 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] with distanceFromRoot: 0
2016-01-14 15:48:02,208 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:02,208 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:02,214 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A79F425B275C462BBC16EF900023880C=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:02,214 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] parallelism set to 1
2016-01-14 15:48:02,214 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:02,216 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786480985_0001_1_01, eventCount=1
2016-01-14 15:48:02,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:02,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, vertexId=vertex_1452786480985_0001_1_01, initRequestedTime=1452786482049, initedTime=1452786482216, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:02,217 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] with distanceFromRoot: 0
2016-01-14 15:48:02,217 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:02,218 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:02,218 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:02,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786480985_0001_1_00, startRequestedTime=1452786482217, startedTime=1452786482217
2016-01-14 15:48:02,224 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:02,224 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:02,224 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:02,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786480985_0001_1_01, startRequestedTime=1452786482224, startedTime=1452786482224
2016-01-14 15:48:02,226 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:02,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:02,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_STARTED]: vertexName=C0B493A9663346E5ADB005D506F37D96, taskId=task_1452786480985_0001_1_00_000000, scheduledTime=1452786482254, launchTime=1452786482254
2016-01-14 15:48:02,255 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:02,256 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786480985_0001_1_00 for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] numStartedSources: 1 numSources: 2
2016-01-14 15:48:02,256 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] numStartedSources: 1 numSources: 2
2016-01-14 15:48:02,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:02,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_STARTED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, taskId=task_1452786480985_0001_1_01_000000, scheduledTime=1452786482257, launchTime=1452786482257
2016-01-14 15:48:02,259 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:02,260 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786480985_0001_1_01 for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] numStartedSources: 2 numSources: 2
2016-01-14 15:48:02,260 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] with distanceFromRoot: 1
2016-01-14 15:48:02,262 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,263 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,267 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AE9CB00F172D424DAA1215E1A113C882 in vertex: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,267 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C0B493A9663346E5ADB005D506F37D96 in vertex: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,270 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C93856C0D38D44EA93187859B133FBD7 with 2 source tasks and 1 pending tasks
2016-01-14 15:48:02,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:02,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786480985_0001_1_02, startRequestedTime=1452786482263, startedTime=1452786482263
2016-01-14 15:48:02,271 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:02,274 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: C0B493A9663346E5ADB005D506F37D96, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B4B66DEC01D946AAAA58980743758C63, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:02,276 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:02,277 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: AE9CB00F172D424DAA1215E1A113C882, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A79F425B275C462BBC16EF900023880C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:02,277 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:02,277 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:02,280 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:02,312 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:02,316 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:02,316 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:02,320 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786480985_0001_00_000001
2016-01-14 15:48:02,320 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:02,321 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786480985_0001_00_000002
2016-01-14 15:48:02,321 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:02,327 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786480985_0001_00_000001
2016-01-14 15:48:02,327 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:02,331 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:02,333 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:02,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786480985_0001_00_000001, launchTime=1452786482332
2016-01-14 15:48:02,336 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000001
2016-01-14 15:48:02,339 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786480985_0001_1_00_000000_0] to container: [container_1452786480985_0001_00_000001]
2016-01-14 15:48:02,340 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:02,340 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786480985_0001_00_000001 given task: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,341 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000001: 1 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02 Starting to run new task attempt: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,341 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,342 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:02,342 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786480985_0001_00_000002
2016-01-14 15:48:02,342 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:02,343 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:02,344 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:02,344 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786480985_0001_1_00_000000_0] started. Is using containerId: [container_1452786480985_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:02,344 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:02,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:02,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C0B493A9663346E5ADB005D506F37D96, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0, startTime=1452786482341, containerId=container_1452786480985_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786480985_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:02,347 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:02,347 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-01-14 15:48:02,355 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:02,355 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:02,355 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:02,356 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:02,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786480985_0001_00_000002, launchTime=1452786482354
2016-01-14 15:48:02,359 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000002
2016-01-14 15:48:02,360 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786480985_0001_1_01_000000_0] to container: [container_1452786480985_0001_00_000002]
2016-01-14 15:48:02,360 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:02,361 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786480985_0001_1_01_000000_0] started. Is using containerId: [container_1452786480985_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:02,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:02,362 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786480985_0001_00_000002 given task: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,362 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02 Starting to run new task attempt: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0, startTime=1452786482361, containerId=container_1452786480985_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786480985_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:02,362 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:02,363 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:02,363 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:02,363 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:02,363 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:02,363 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:02,364 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:02,380 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:02,380 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:02,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: AE9CB00F172D424DAA1215E1A113C882, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A79F425B275C462BBC16EF900023880C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:02,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: C0B493A9663346E5ADB005D506F37D96, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B4B66DEC01D946AAAA58980743758C63, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:02,383 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:02,383 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:02,388 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:02,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,393 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:02,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B4B66DEC01D946AAAA58980743758C63, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:02,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:02,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:02,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:02,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A79F425B275C462BBC16EF900023880C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:02,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,409 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786480985_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:02,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:02,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:02,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:02,409 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786480985_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:02,415 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,419 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,420 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:02,430 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:02,435 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,434 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:02,430 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:02,443 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,442 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:02,442 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:02,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,447 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,448 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,466 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C0B493A9663346E5ADB005D506F37D96, ordinal: 1
2016-01-14 15:48:02,466 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AE9CB00F172D424DAA1215E1A113C882, ordinal: 0
2016-01-14 15:48:02,468 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0B493A9663346E5ADB005D506F37D96, mem on start (mb), free: 234, total: 281, max: 672
2016-01-14 15:48:02,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,468 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE9CB00F172D424DAA1215E1A113C882, mem on start (mb), free: 234, total: 281, max: 672
2016-01-14 15:48:02,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:02,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:02,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:02,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:02,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:02,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:02,471 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:02,471 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:02,478 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:02,481 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:02,482 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:02,482 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:02,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:02,484 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A79F425B275C462BBC16EF900023880C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:02,484 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C93856C0D38D44EA93187859B133FBD7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:02,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:02,479 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:02,486 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:02,487 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:02,487 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:02,487 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:02,488 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B4B66DEC01D946AAAA58980743758C63, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:02,488 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C93856C0D38D44EA93187859B133FBD7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:02,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:02,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,489 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:02,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B4B66DEC01D946AAAA58980743758C63 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:02,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,491 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:02,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A79F425B275C462BBC16EF900023880C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:02,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:02,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:02,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:02,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:02,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:02,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:02,495 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,498 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,497 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:02,501 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:02,515 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@438b5662
2016-01-14 15:48:02,515 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@70d45e92
2016-01-14 15:48:02,532 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@70d45e92
2016-01-14 15:48:02,533 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:02,533 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:02,537 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@438b5662
2016-01-14 15:48:02,537 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:02,537 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:02,542 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:02,551 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:02,580 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,580 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:02,581 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:02,582 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"] streamed: true, id: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,582 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:02,583 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:02,602 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@3205d005; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@235c6976; comparator=cascading.tuple.hadoop.util.TupleComparator@80fd41d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:02,602 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1c91f372; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1b9c11eb; comparator=cascading.tuple.hadoop.util.TupleComparator@472ec00d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:02,605 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:02,606 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:02,607 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:02,608 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:02,609 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:02,609 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:02,730 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:02,731 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:02,731 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:02,731 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:02,731 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:02,739 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:02,739 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:02,742 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C0B493A9663346E5ADB005D506F37D96, all 1 inputs ready in: 00:00:00.001
2016-01-14 15:48:02,748 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:02,749 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:02,749 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:02,749 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:02,749 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:02,750 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:02,752 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:02,752 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AE9CB00F172D424DAA1215E1A113C882, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:02,757 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,757 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE9CB00F172D424DAA1215E1A113C882, mem on close (mb), free: 121, total: 382, max: 672
2016-01-14 15:48:02,757 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:02,757 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:02,757 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 102; bufvoid = 104857600
2016-01-14 15:48:02,758 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 15:48:02,792 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,792 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0B493A9663346E5ADB005D506F37D96, mem on close (mb), free: 115, total: 382, max: 672
2016-01-14 15:48:02,792 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:02,793 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:02,793 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 9061; bufvoid = 104857600
2016-01-14 15:48:02,793 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2016-01-14 15:48:05,010 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:05 Completed running task attempt: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:05,035 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:05,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786480985_0001_1_01, eventCount=1
2016-01-14 15:48:05,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=174, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=99, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=501219328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786482752, Process_Duration=5, Process_End_Time=1452786482757, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 15:48:05,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:05,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0, startTime=1452786482361, finishTime=1452786485037, timeTaken=2676, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=174, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=99, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=501219328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482752, Process_Duration=5, Process_End_Time=1452786482757, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:48:05,043 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:05,043 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:05,045 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:05,046 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:05,046 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000002
2016-01-14 15:48:05,047 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:05,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:05,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_FINISHED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, taskId=task_1452786480985_0001_1_01_000000, startTime=1452786482361, finishTime=1452786485048, timeTaken=2687, status=SUCCEEDED, successfulAttemptID=attempt_1452786480985_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=174, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=99, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=501219328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482752, Process_Duration=5, Process_End_Time=1452786482757, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:48:05,054 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:05,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:05,055 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] : 1
2016-01-14 15:48:05,055 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:05,069 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:05,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_FINISHED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, vertexId=vertex_1452786480985_0001_1_01, initRequestedTime=1452786482049, initedTime=1452786482216, startRequestedTime=1452786482224, startedTime=1452786482224, finishTime=1452786485055, timeTaken=2831, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=174, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=99, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=501219328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482752, Process_Duration=5, Process_End_Time=1452786482757, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452786482361, firstTasksToStart=[ task_1452786480985_0001_1_01_000000 ], lastTaskFinishTime=1452786485037, lastTasksToFinish=[ task_1452786480985_0001_1_01_000000 ], minTaskDuration=2676, maxTaskDuration=2676, avgTaskDuration=2676.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786480985_0001_1_01_000000 ], longestDurationTasks=[ task_1452786480985_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:05,078 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:05,080 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786480985_0001_00_000002, stoppedTime=1452786485080, exitStatus=0
2016-01-14 15:48:05,081 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:05,080 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:05,082 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] attempt: attempt_1452786480985_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:05,085 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:05,085 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786480985_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:48:05,085 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:05,107 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786480985_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:05,108 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000002: 62 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:05,108 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786480985_0001_00_000002, Exiting
2016-01-14 15:48:05,109 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786480985_0001_00_000002
2016-01-14 15:48:05,109 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786480985_0001_00_000002
2016-01-14 15:48:05,111 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786480985_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:05,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786480985_0001_00_000002 completed successfully
2016-01-14 15:48:05,112 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:18,349 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:18 Completed running task attempt: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:18,354 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:18,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786480985_0001_1_00, eventCount=1
2016-01-14 15:48:18,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17077, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=200, GC_TIME_MILLIS=250, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786482742, Process_Duration=50, Process_End_Time=1452786482792, Read_Duration=5, Tuples_Read=200, Tuples_Written=200, Write_Duration=13][cascading.flow.StepCounters Tuples_Read=200]]
2016-01-14 15:48:18,355 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:18,356 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:18,357 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:18,358 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000001
2016-01-14 15:48:18,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C0B493A9663346E5ADB005D506F37D96, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0, startTime=1452786482341, finishTime=1452786498355, timeTaken=16014, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17077, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=250, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482742, Process_Duration=50, Process_End_Time=1452786482792, Read_Duration=5, Tuples_Read=200, Tuples_Written=200, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=200
2016-01-14 15:48:18,359 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:18,360 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:18,360 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:18,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:18,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_FINISHED]: vertexName=C0B493A9663346E5ADB005D506F37D96, taskId=task_1452786480985_0001_1_00_000000, startTime=1452786482341, finishTime=1452786498360, timeTaken=16019, status=SUCCEEDED, successfulAttemptID=attempt_1452786480985_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17077, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=250, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482742, Process_Duration=50, Process_End_Time=1452786482792, Read_Duration=5, Tuples_Read=200, Tuples_Written=200, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=200
2016-01-14 15:48:18,362 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:18,362 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] : 1
2016-01-14 15:48:18,362 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:18,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:18,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_FINISHED]: vertexName=C0B493A9663346E5ADB005D506F37D96, vertexId=vertex_1452786480985_0001_1_00, initRequestedTime=1452786482026, initedTime=1452786482206, startRequestedTime=1452786482217, startedTime=1452786482217, finishTime=1452786498364, timeTaken=16147, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17077, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=250, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482742, Process_Duration=50, Process_End_Time=1452786482792, Read_Duration=5, Tuples_Read=200, Tuples_Written=200, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=200, vertexStats=firstTaskStartTime=1452786482341, firstTasksToStart=[ task_1452786480985_0001_1_00_000000 ], lastTaskFinishTime=1452786498355, lastTasksToFinish=[ task_1452786480985_0001_1_00_000000 ], minTaskDuration=16014, maxTaskDuration=16014, avgTaskDuration=16014.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786480985_0001_1_00_000000 ], longestDurationTasks=[ task_1452786480985_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:18,367 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:18,367 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:18,368 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] attempt: attempt_1452786480985_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:18,368 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:18,368 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:18,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:18,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786480985_0001_00_000001, stoppedTime=1452786498370, exitStatus=0
2016-01-14 15:48:18,371 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:18,371 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:18,372 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:18,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_STARTED]: vertexName=C93856C0D38D44EA93187859B133FBD7, taskId=task_1452786480985_0001_1_02_000000, scheduledTime=1452786498372, launchTime=1452786498372
2016-01-14 15:48:18,373 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:18,373 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786480985_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:18,374 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:18,374 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:18,375 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: C93856C0D38D44EA93187859B133FBD7, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AE9CB00F172D424DAA1215E1A113C882, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C0B493A9663346E5ADB005D506F37D96, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=30089AFED8C24CFF9C76F292A1703C4B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:18,375 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:18,375 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:18,376 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786480985_0001_00_000003
2016-01-14 15:48:18,377 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:18,379 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786480985_0001_00_000003
2016-01-14 15:48:18,379 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:18,382 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:18,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786480985_0001_00_000003, launchTime=1452786498382
2016-01-14 15:48:18,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:18,383 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000003
2016-01-14 15:48:18,384 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786480985_0001_1_02_000000_0] to container: [container_1452786480985_0001_00_000003]
2016-01-14 15:48:18,384 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:18,385 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786480985_0001_1_02_000000_0] started. Is using containerId: [container_1452786480985_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 15:48:18,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C93856C0D38D44EA93187859B133FBD7, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0, startTime=1452786498385, containerId=container_1452786480985_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786480985_0001_00_000003/teamcity, completedLogs=
2016-01-14 15:48:18,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:18,385 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:18,386 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:18,386 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786480985_0001_00_000003 given task: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,386 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18 Starting to run new task attempt: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,387 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,387 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:18,387 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:18,387 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:18,387 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:18,387 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:18,387 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:18,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: C93856C0D38D44EA93187859B133FBD7, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AE9CB00F172D424DAA1215E1A113C882, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C0B493A9663346E5ADB005D506F37D96, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=30089AFED8C24CFF9C76F292A1703C4B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:18,388 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:18,388 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:18,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:18,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AE9CB00F172D424DAA1215E1A113C882, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:18,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C0B493A9663346E5ADB005D506F37D96, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:18,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:18,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:18,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:18,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=30089AFED8C24CFF9C76F292A1703C4B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:18,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:18,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 30089AFED8C24CFF9C76F292A1703C4B
2016-01-14 15:48:18,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,393 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:18,394 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 30089AFED8C24CFF9C76F292A1703C4B
2016-01-14 15:48:18,394 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:18,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,397 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786480985_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:18,397 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000001: 38 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:18,397 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786480985_0001_00_000001, Exiting
2016-01-14 15:48:18,397 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786480985_0001_00_000001
2016-01-14 15:48:18,397 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786480985_0001_00_000001
2016-01-14 15:48:18,412 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786480985_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:18,412 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786480985_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 15:48:18,412 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786480985_0001_00_000001 completed successfully
2016-01-14 15:48:18,413 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:18,414 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C93856C0D38D44EA93187859B133FBD7, ordinal: 2
2016-01-14 15:48:18,414 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C93856C0D38D44EA93187859B133FBD7, mem on start (mb), free: 124, total: 493, max: 672
2016-01-14 15:48:18,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:18,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 15:48:18,419 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:18,421 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:18,421 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,419 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:18,422 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:18,422 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:18,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:18,429 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:18,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 30089AFED8C24CFF9C76F292A1703C4B
2016-01-14 15:48:18,430 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:18,430 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:18,430 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:18,430 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:18,430 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:18,430 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:18,431 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 15:48:18,431 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 15:48:18,431 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 30089AFED8C24CFF9C76F292A1703C4B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:18,431 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AE9CB00F172D424DAA1215E1A113C882, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:48:18,431 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C0B493A9663346E5ADB005D506F37D96, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:48:18,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:18,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AE9CB00F172D424DAA1215E1A113C882 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:18,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C0B493A9663346E5ADB005D506F37D96 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:18,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 15:48:18,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 15:48:18,440 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:18,440 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:18,441 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:18,441 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:18,445 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C0B493A9663346E5ADB005D506F37D96 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:18,445 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AE9CB00F172D424DAA1215E1A113C882 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:18,448 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:48:18,448 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:18,448 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:48:18,448 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:18,452 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:18,452 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:18,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:18,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:18,457 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,459 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786480985_0001_1_01_000000_0_10006, runDuration: 2598164, ]
2016-01-14 15:48:18,462 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786480985_0001_1_00_000000_0_10004, runDuration: 15931612, ]
2016-01-14 15:48:18,465 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2016-01-14 15:48:18,465 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786480985_0001_1_01_000000_0_10006] to DISK_DIRECT, CompressedSize=142, DecompressedSize=138,EndTime=1452786498465, TimeTaken=2, Rate=0.07 MB/s
2016-01-14 15:48:18,465 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786480985_0001_1_00_000000_0_10004] to DISK_DIRECT, CompressedSize=8931, DecompressedSize=8927,EndTime=1452786498465, TimeTaken=2, Rate=4.26 MB/s
2016-01-14 15:48:18,466 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,466 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,466 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s)
2016-01-14 15:48:18,467 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C0B493A9663346E5ADB005D506F37D96] #2 in 4ms
2016-01-14 15:48:18,467 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C0B493A9663346E5ADB005D506F37D96] #2, status:false, isInterrupted:false
2016-01-14 15:48:18,468 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:18,468 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:18,469 WARN util.TupleHasher (TupleHasher.java:merge(73)) - two unequal Hasher instances for the same key field position found: cascading.platform.hadoop.TestLongComparator@7e308c04, and: cascading.platform.hadoop.TestLongComparator@25389b55
2016-01-14 15:48:18,470 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] streamed: true, id: 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:18,470 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"], id: 30089AFED8C24CFF9C76F292A1703C4B
2016-01-14 15:48:18,470 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"]
2016-01-14 15:48:18,470 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 8931 bytes from disk
2016-01-14 15:48:18,471 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AE9CB00F172D424DAA1215E1A113C882] #1 in 8ms
2016-01-14 15:48:18,471 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:18,471 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AE9CB00F172D424DAA1215E1A113C882] #1, status:false, isInterrupted:false
2016-01-14 15:48:18,472 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:18,473 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 142 bytes from disk
2016-01-14 15:48:18,473 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:18,473 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 4B42A92D7FD54B7C99C3C46A0070B663, for 2 inputs
2016-01-14 15:48:18,474 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 4B42A92D7FD54B7C99C3C46A0070B663, for 2 inputs
2016-01-14 15:48:18,478 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:18,478 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:18,481 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:48:18,481 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:48:18,487 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 8917 bytes
2016-01-14 15:48:18,487 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,488 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 128 bytes
2016-01-14 15:48:18,488 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,489 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:18,489 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C93856C0D38D44EA93187859B133FBD7, all 2 inputs ready in: 00:00:00.000
2016-01-14 15:48:18,490 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:18,494 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@76fc3212; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@273b78c5
2016-01-14 15:48:18,496 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3175e1a2; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@32306c0c
2016-01-14 15:48:18,526 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:18,526 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:18,526 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,527 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786480985_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 15:48:18,528 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,528 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C93856C0D38D44EA93187859B133FBD7, mem on close (mb), free: 112, total: 493, max: 672
2016-01-14 15:48:18,528 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,529 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:18,529 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:18,530 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:18,530 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:18,530 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:18,531 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,531 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:18,532 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:18,532 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:18,532 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:18,532 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:18 Completed running task attempt: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,534 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:18,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:18,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786498490, Process_Duration=38, Process_End_Time=1452786498528, Read_Duration=13, Tuples_Read=124, Tuples_Written=199, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=199]]
2016-01-14 15:48:18,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C93856C0D38D44EA93187859B133FBD7, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0, startTime=1452786498385, finishTime=1452786498536, timeTaken=151, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786498490, Process_Duration=38, Process_End_Time=1452786498528, Read_Duration=13, Tuples_Read=124, Tuples_Written=199, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=199
2016-01-14 15:48:18,538 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:18,538 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:18,539 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:18,540 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:18,538 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:18,541 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000003
2016-01-14 15:48:18,543 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_FINISHED]: vertexName=C93856C0D38D44EA93187859B133FBD7, taskId=task_1452786480985_0001_1_02_000000, startTime=1452786498385, finishTime=1452786498540, timeTaken=155, status=SUCCEEDED, successfulAttemptID=attempt_1452786480985_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786498490, Process_Duration=38, Process_End_Time=1452786498528, Read_Duration=13, Tuples_Read=124, Tuples_Written=199, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=199
2016-01-14 15:48:18,543 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:18,543 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] : 1
2016-01-14 15:48:18,543 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:18,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:18,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_FINISHED]: vertexName=C93856C0D38D44EA93187859B133FBD7, vertexId=vertex_1452786480985_0001_1_02, initRequestedTime=1452786482052, initedTime=1452786482177, startRequestedTime=1452786482263, startedTime=1452786482263, finishTime=1452786498543, timeTaken=16280, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786498490, Process_Duration=38, Process_End_Time=1452786498528, Read_Duration=13, Tuples_Read=124, Tuples_Written=199, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=199, vertexStats=firstTaskStartTime=1452786498385, firstTasksToStart=[ task_1452786480985_0001_1_02_000000 ], lastTaskFinishTime=1452786498536, lastTasksToFinish=[ task_1452786480985_0001_1_02_000000 ], minTaskDuration=151, maxTaskDuration=151, avgTaskDuration=151.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786480985_0001_1_02_000000 ], longestDurationTasks=[ task_1452786480985_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:18,546 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:18,546 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:18,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786480985_0001_00_000003, stoppedTime=1452786498547, exitStatus=0
2016-01-14 15:48:18,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:18,547 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:18,547 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:18,548 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786480985_0001_1
2016-01-14 15:48:18,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786480985_0001_1
2016-01-14 15:48:18,549 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 30089AFED8C24CFF9C76F292A1703C4B for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:18,552 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786480985_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 15:48:18,552 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:18,553 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786480985_0001_00_000003, Exiting
2016-01-14 15:48:18,553 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786480985_0001_00_000003
2016-01-14 15:48:18,553 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786480985_0001_00_000003
2016-01-14 15:48:18,553 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786480985_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:18,554 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786480985_0001_00_000003 completed successfully
2016-01-14 15:48:18,560 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:18,561 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:18,567 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786480985_0001_1, queueSize=0
2016-01-14 15:48:18,568 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:18,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786480985_0001_1, startTime=1452786482023, finishTime=1452786498561, timeTaken=16538, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=83619, FILE_BYTES_WRITTEN=23910, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=440, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=349, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1535115264, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=419, OUTPUT_BYTES=9163, OUTPUT_BYTES_WITH_OVERHEAD=9065, OUTPUT_BYTES_PHYSICAL=9073, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358359463984, Process_Duration=93, Process_End_Time=4358359464077, Read_Duration=21, Tuples_Read=344, Tuples_Written=419, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=220, Tuples_Written=199
2016-01-14 15:48:18,569 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786480985_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:18,569 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786480985_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:18,570 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786480985_0001_00_000003
2016-01-14 15:48:18,570 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:18 Completed Dag: dag_1452786480985_0001_1
2016-01-14 15:48:18,572 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786480985_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:18,572 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786480985_0001_1_post
2016-01-14 15:48:18,573 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:18,573 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:18,578 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1452786480985_0001
2016-01-14 15:48:18,581 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:18,581 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:18,581 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:18,581 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:18,581 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:18,582 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:18,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:18,585 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:18,585 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:18,585 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:18,585 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:18,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:18,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:18,591 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001
2016-01-14 15:48:18,591 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] completed in: 00:17.809
2016-01-14 15:48:18,653 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed
2016-01-14 15:48:18,685 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:18,686 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:18,758 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-01-14 15:48:18,803 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.088
2016-01-14 15:48:18,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:48:18,805 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:48:18,806 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:18,821 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 15:48:18,838 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt
2016-01-14 15:48:18,856 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward
2016-01-14 15:48:18,942 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] starting
2016-01-14 15:48:18,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:18,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:18,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"]
2016-01-14 15:48:18,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] parallel execution of steps is enabled: false
2016-01-14 15:48:18,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executing total steps: 1
2016-01-14 15:48:18,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] allocating management threads: 1
2016-01-14 15:48:18,945 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] starting step: (1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:18,947 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [apache*ip] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:18,956 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tez session mode enabled: true
2016-01-14 15:48:18,962 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:18,963 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:18,963 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:18,973 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001 doesn't exist and is created
2016-01-14 15:48:18,992 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:18,993 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:18,994 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001
2016-01-14 15:48:18,998 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786498963_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:19,015 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:19,017 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:19,020 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:19,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:19,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001/localmode-log-dir
2016-01-14 15:48:19,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001/localmode-log-dir/history.txt.appattempt_1452786498963_0001_000000, maxErrors=10
2016-01-14 15:48:19,021 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:19,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786498963_0001_000000, appSubmitTime=1452786498997, launchTime=1452786498997
2016-01-14 15:48:19,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:19,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:19,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:19,036 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:45252
2016-01-14 15:48:19,037 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:19,040 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:19,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786498963_0001_000000, startTime=1452786499042
2016-01-14 15:48:19,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:19,042 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:19,093 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:19,093 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:19,093 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1452786498963_0001, dagName=[8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:19,110 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:19,113 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786498963_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001/localmode-log-dir/dag_1452786498963_0001_1.dot
2016-01-14 15:48:19,114 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001/localmode-log-dir/dag_1452786498963_0001_1-tez-dag.pb.txt
2016-01-14 15:48:19,146 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786498963_0001_1
2016-01-14 15:48:19,146 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:19,147 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:19 Running Dag: dag_1452786498963_0001_1
2016-01-14 15:48:19,164 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786498963_0001_1, submitTime=1452786499110
2016-01-14 15:48:19,164 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:19,166 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,166 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:19,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786498963_0001_1, initTime=1452786499164
2016-01-14 15:48:19,167 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786498963_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:19,166 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:19,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_STARTED]: dagID=dag_1452786498963_0001_1, startTime=1452786499168
2016-01-14 15:48:19,169 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:19,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:19,169 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786498963_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:19,170 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E2E2D7DC0F3041E7ABC17AC0FA602344 : {3EA1543B792642C5878D8BFD49565491={InputName=3EA1543B792642C5878D8BFD49565491}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:19,170 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3EA1543B792642C5878D8BFD49565491, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:19,170 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,171 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,171 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,171 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1452786498963_0001, dagName=[8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:19,171 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]. Starting root input initializers: 1
2016-01-14 15:48:19,172 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:19,172 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3EA1543B792642C5878D8BFD49565491 on vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,172 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D262ACA9B2A94E9E8430EB2120B6B3A9 : {2E0F3C807AC941DB9890A121B97B8186={InputName=2E0F3C807AC941DB9890A121B97B8186}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:19,172 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2E0F3C807AC941DB9890A121B97B8186, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:19,172 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,173 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,173 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,173 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]. Starting root input initializers: 1
2016-01-14 15:48:19,174 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:19,174 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,174 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2E0F3C807AC941DB9890A121B97B8186 on vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,176 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:19,177 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3EA1543B792642C5878D8BFD49565491 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:19,179 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:19,179 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:19,182 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 410
2016-01-14 15:48:19,185 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3EA1543B792642C5878D8BFD49565491 on vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,183 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:19,186 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,186 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,187 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,187 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B71CA62645984090B1A2804D20C09E5D, vertexId=vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:19,188 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B71CA62645984090B1A2804D20C09E5D, vertexId=vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,191 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:19,191 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2E0F3C807AC941DB9890A121B97B8186 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:19,194 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:19,195 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:19,193 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B71CA62645984090B1A2804D20C09E5D, vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:19,198 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B71CA62645984090B1A2804D20C09E5D, vertexId=vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] submitted tez dag to app master: application_1452786498963_0001, with dag id: dag_1452786498963_0001_1
2016-01-14 15:48:19,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, vertexId=vertex_1452786498963_0001_1_02, initRequestedTime=1452786499174, initedTime=1452786499200, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:19,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:19,202 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 15:48:19,203 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:19,204 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:19,204 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3EA1543B792642C5878D8BFD49565491=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:19,204 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] parallelism set to 1
2016-01-14 15:48:19,204 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:19,203 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2E0F3C807AC941DB9890A121B97B8186 on vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786498963_0001_1_00, eventCount=1
2016-01-14 15:48:19,207 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:19,208 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, vertexId=vertex_1452786498963_0001_1_00, initRequestedTime=1452786499170, initedTime=1452786499207, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:19,208 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] with distanceFromRoot: 0
2016-01-14 15:48:19,208 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:19,209 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:19,209 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2E0F3C807AC941DB9890A121B97B8186=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:19,211 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] parallelism set to 1
2016-01-14 15:48:19,211 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:19,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786498963_0001_1_01, eventCount=1
2016-01-14 15:48:19,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, vertexId=vertex_1452786498963_0001_1_01, initRequestedTime=1452786499172, initedTime=1452786499212, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:19,213 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] with distanceFromRoot: 0
2016-01-14 15:48:19,213 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:19,213 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:19,214 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:19,214 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786498963_0001_1_00, startRequestedTime=1452786499214, startedTime=1452786499214
2016-01-14 15:48:19,215 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:19,216 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:19,216 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:19,216 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:19,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786498963_0001_1_01, startRequestedTime=1452786499216, startedTime=1452786499216
2016-01-14 15:48:19,217 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:19,218 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:19,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_STARTED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, taskId=task_1452786498963_0001_1_00_000000, scheduledTime=1452786499218, launchTime=1452786499218
2016-01-14 15:48:19,219 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:19,219 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786498963_0001_1_00 for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] numStartedSources: 1 numSources: 2
2016-01-14 15:48:19,220 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] numStartedSources: 1 numSources: 2
2016-01-14 15:48:19,220 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:19,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_STARTED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, taskId=task_1452786498963_0001_1_01_000000, scheduledTime=1452786499220, launchTime=1452786499220
2016-01-14 15:48:19,221 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:19,221 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786498963_0001_1_01 for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] numStartedSources: 2 numSources: 2
2016-01-14 15:48:19,221 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] with distanceFromRoot: 1
2016-01-14 15:48:19,221 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,222 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,222 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D262ACA9B2A94E9E8430EB2120B6B3A9 in vertex: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,222 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E2E2D7DC0F3041E7ABC17AC0FA602344 in vertex: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,222 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A0C94E7A8AEA45F0A8819AD7A58541D3 with 2 source tasks and 1 pending tasks
2016-01-14 15:48:19,223 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:19,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786498963_0001_1_02, startRequestedTime=1452786499222, startedTime=1452786499222
2016-01-14 15:48:19,223 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:19,223 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: E2E2D7DC0F3041E7ABC17AC0FA602344, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3EA1543B792642C5878D8BFD49565491, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:19,224 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:19,224 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: D262ACA9B2A94E9E8430EB2120B6B3A9, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2E0F3C807AC941DB9890A121B97B8186, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:19,224 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:19,225 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:19,225 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:19,225 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:19,226 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:19,226 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:19,226 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786498963_0001_00_000001
2016-01-14 15:48:19,226 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:19,227 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786498963_0001_00_000002
2016-01-14 15:48:19,227 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:19,229 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786498963_0001_00_000001
2016-01-14 15:48:19,229 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:19,230 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:19,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:19,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786498963_0001_00_000001, launchTime=1452786499230
2016-01-14 15:48:19,232 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000001
2016-01-14 15:48:19,233 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786498963_0001_1_00_000000_0] to container: [container_1452786498963_0001_00_000001]
2016-01-14 15:48:19,233 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:19,234 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786498963_0001_00_000001 given task: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,234 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786498963_0001_1_00_000000_0] started. Is using containerId: [container_1452786498963_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:19,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0, startTime=1452786499234, containerId=container_1452786498963_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786498963_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:19,234 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:19,235 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786498963_0001_00_000002
2016-01-14 15:48:19,235 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:19,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:19,234 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:19,235 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:19,236 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:19,236 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786498963_0001_00_000002, launchTime=1452786499235
2016-01-14 15:48:19,237 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000002
2016-01-14 15:48:19 Starting to run new task attempt: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,239 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,240 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:19,240 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:19,240 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:19,240 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:19,240 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:19,242 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:19,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: E2E2D7DC0F3041E7ABC17AC0FA602344, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3EA1543B792642C5878D8BFD49565491, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:19,242 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:19,241 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786498963_0001_1_01_000000_0] to container: [container_1452786498963_0001_00_000002]
2016-01-14 15:48:19,243 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:19,244 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786498963_0001_00_000002 given task: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,244 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,244 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786498963_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:19 Starting to run new task attempt: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:19,245 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,247 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:19,247 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:19,247 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:19,247 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:19,247 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:19,247 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:19,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: D262ACA9B2A94E9E8430EB2120B6B3A9, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2E0F3C807AC941DB9890A121B97B8186, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:19,245 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786498963_0001_1_01_000000_0] started. Is using containerId: [container_1452786498963_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:19,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0, startTime=1452786499244, containerId=container_1452786498963_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786498963_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:19,249 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:19,249 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:19,249 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:19,250 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:19,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,253 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:19,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3EA1543B792642C5878D8BFD49565491, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:19,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:19,257 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786498963_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:19,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2E0F3C807AC941DB9890A121B97B8186, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:19,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:19,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:19,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:19,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:19,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,262 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:19,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,264 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,260 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:19,264 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:19,264 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,266 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:19,272 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:19,272 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,275 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:19,275 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,277 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:19,278 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,288 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D262ACA9B2A94E9E8430EB2120B6B3A9, ordinal: 0
2016-01-14 15:48:19,288 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D262ACA9B2A94E9E8430EB2120B6B3A9, mem on start (mb), free: 203, total: 493, max: 672
2016-01-14 15:48:19,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:19,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:19,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:19,289 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:19,289 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:19,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:19,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:19,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:19,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:19,291 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2E0F3C807AC941DB9890A121B97B8186, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:19,291 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A0C94E7A8AEA45F0A8819AD7A58541D3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:19,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:19,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2E0F3C807AC941DB9890A121B97B8186 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:19,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,292 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:19,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,293 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:19,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:19,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:19,295 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,297 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:19,300 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7a223f56
2016-01-14 15:48:19,301 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E2E2D7DC0F3041E7ABC17AC0FA602344, ordinal: 1
2016-01-14 15:48:19,302 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E2E2D7DC0F3041E7ABC17AC0FA602344, mem on start (mb), free: 201, total: 493, max: 672
2016-01-14 15:48:19,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:19,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:19,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:19,306 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:19,306 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:19,307 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:19,308 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:19,309 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:19,309 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:19,310 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3EA1543B792642C5878D8BFD49565491, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:19,311 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A0C94E7A8AEA45F0A8819AD7A58541D3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:19,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:19,305 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7a223f56
2016-01-14 15:48:19,305 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:19,312 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:19,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,313 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3EA1543B792642C5878D8BFD49565491 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:19,314 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:19,315 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,314 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:19,315 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:19,316 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:19,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:19,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:19,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:19,318 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1031a99a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2c75c5c4; comparator=cascading.tuple.hadoop.util.TupleComparator@680adda9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:19,319 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:19,320 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@516873e1
2016-01-14 15:48:19,321 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@516873e1
2016-01-14 15:48:19,319 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,319 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:19,322 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:19,323 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:19,323 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:19,323 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:19,337 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:19,338 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:19,338 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:19,339 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:19,339 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:19,340 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:19,339 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"] streamed: true, id: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,340 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:19,340 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:19,341 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:19,341 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:19,341 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D262ACA9B2A94E9E8430EB2120B6B3A9, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:19,345 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1ba90cc; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7f7841f6; comparator=cascading.tuple.hadoop.util.TupleComparator@3032bedb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:19,345 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:19,346 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:19,346 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,346 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D262ACA9B2A94E9E8430EB2120B6B3A9, mem on close (mb), free: 99, total: 493, max: 672
2016-01-14 15:48:19,347 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:19,347 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:19,347 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 102; bufvoid = 104857600
2016-01-14 15:48:19,347 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 15:48:19,348 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:19,823 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:19,823 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:19,823 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:19,823 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:19,823 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:19,832 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:19,832 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:19,832 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E2E2D7DC0F3041E7ABC17AC0FA602344, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:19,860 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,861 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E2E2D7DC0F3041E7ABC17AC0FA602344, mem on close (mb), free: 292, total: 520, max: 672
2016-01-14 15:48:19,861 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:19,861 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:19,861 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 9061; bufvoid = 104857600
2016-01-14 15:48:19,861 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2016-01-14 15:48:21,026 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:21 Completed running task attempt: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:21,031 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:21,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786498963_0001_1_01, eventCount=1
2016-01-14 15:48:21,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:21,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0, startTime=1452786499244, finishTime=1452786501032, timeTaken=1788, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=694, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=393, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=545783808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499342, Process_Duration=4, Process_End_Time=1452786499346, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:48:21,033 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:21,033 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:21,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_FINISHED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, taskId=task_1452786498963_0001_1_01_000000, startTime=1452786499244, finishTime=1452786501034, timeTaken=1790, status=SUCCEEDED, successfulAttemptID=attempt_1452786498963_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=694, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=393, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=545783808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499342, Process_Duration=4, Process_End_Time=1452786499346, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:48:21,034 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:21,034 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] : 1
2016-01-14 15:48:21,034 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:21,035 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:21,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=694, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=393, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=545783808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786499342, Process_Duration=4, Process_End_Time=1452786499346, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 15:48:21,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:21,043 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:21,043 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:21,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_FINISHED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, vertexId=vertex_1452786498963_0001_1_01, initRequestedTime=1452786499172, initedTime=1452786499212, startRequestedTime=1452786499216, startedTime=1452786499216, finishTime=1452786501034, timeTaken=1818, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=694, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=393, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=545783808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499342, Process_Duration=4, Process_End_Time=1452786499346, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452786499244, firstTasksToStart=[ task_1452786498963_0001_1_01_000000 ], lastTaskFinishTime=1452786501032, lastTasksToFinish=[ task_1452786498963_0001_1_01_000000 ], minTaskDuration=1788, maxTaskDuration=1788, avgTaskDuration=1788.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786498963_0001_1_01_000000 ], longestDurationTasks=[ task_1452786498963_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:21,044 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:21,044 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] attempt: attempt_1452786498963_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:21,044 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:21,045 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000002
2016-01-14 15:48:21,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:21,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786498963_0001_00_000002, stoppedTime=1452786501045, exitStatus=0
2016-01-14 15:48:21,047 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:21,047 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:21,047 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786498963_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:48:21,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:21,047 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:21,048 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786498963_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:21,048 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:21,048 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786498963_0001_00_000002, Exiting
2016-01-14 15:48:21,048 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786498963_0001_00_000002
2016-01-14 15:48:21,049 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786498963_0001_00_000002
2016-01-14 15:48:21,052 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786498963_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:21,052 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786498963_0001_00_000002 completed successfully
2016-01-14 15:48:21,053 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:23,583 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:23,583 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:32,291 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:32 Completed running task attempt: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:32,294 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:32,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786498963_0001_1_00, eventCount=1
2016-01-14 15:48:32,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17750, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=200, GC_TIME_MILLIS=534, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786499833, Process_Duration=27, Process_End_Time=1452786499860, Read_Duration=2, Tuples_Read=200, Tuples_Written=200, Write_Duration=9][cascading.flow.StepCounters Tuples_Read=200]]
2016-01-14 15:48:32,295 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:32,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:32,296 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0, startTime=1452786499234, finishTime=1452786512295, timeTaken=13061, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17750, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=534, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499833, Process_Duration=27, Process_End_Time=1452786499860, Read_Duration=2, Tuples_Read=200, Tuples_Written=200, Write_Duration=9, cascading.flow.StepCounters, Tuples_Read=200
2016-01-14 15:48:32,297 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000001
2016-01-14 15:48:32,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:32,297 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:32,297 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:32,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:32,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_FINISHED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, taskId=task_1452786498963_0001_1_00_000000, startTime=1452786499234, finishTime=1452786512298, timeTaken=13064, status=SUCCEEDED, successfulAttemptID=attempt_1452786498963_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17750, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=534, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499833, Process_Duration=27, Process_End_Time=1452786499860, Read_Duration=2, Tuples_Read=200, Tuples_Written=200, Write_Duration=9, cascading.flow.StepCounters, Tuples_Read=200
2016-01-14 15:48:32,298 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:32,299 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] : 1
2016-01-14 15:48:32,299 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:32,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:32,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_FINISHED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, vertexId=vertex_1452786498963_0001_1_00, initRequestedTime=1452786499170, initedTime=1452786499207, startRequestedTime=1452786499214, startedTime=1452786499214, finishTime=1452786512299, timeTaken=13085, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17750, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=534, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499833, Process_Duration=27, Process_End_Time=1452786499860, Read_Duration=2, Tuples_Read=200, Tuples_Written=200, Write_Duration=9, cascading.flow.StepCounters, Tuples_Read=200, vertexStats=firstTaskStartTime=1452786499234, firstTasksToStart=[ task_1452786498963_0001_1_00_000000 ], lastTaskFinishTime=1452786512295, lastTasksToFinish=[ task_1452786498963_0001_1_00_000000 ], minTaskDuration=13061, maxTaskDuration=13061, avgTaskDuration=13061.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786498963_0001_1_00_000000 ], longestDurationTasks=[ task_1452786498963_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:32,301 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:32,301 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:32,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786498963_0001_00_000001, stoppedTime=1452786512301, exitStatus=0
2016-01-14 15:48:32,301 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:32,302 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] attempt: attempt_1452786498963_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:32,302 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:32,302 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:32,303 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:32,303 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786498963_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:32,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:32,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_STARTED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, taskId=task_1452786498963_0001_1_02_000000, scheduledTime=1452786512303, launchTime=1452786512303
2016-01-14 15:48:32,304 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:32,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:32,304 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:32,305 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:32,305 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: A0C94E7A8AEA45F0A8819AD7A58541D3, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B71CA62645984090B1A2804D20C09E5D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:32,305 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:32,306 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:32,306 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786498963_0001_00_000003
2016-01-14 15:48:32,307 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:32,307 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786498963_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:32,307 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:32,308 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786498963_0001_00_000001, Exiting
2016-01-14 15:48:32,308 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786498963_0001_00_000001
2016-01-14 15:48:32,308 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786498963_0001_00_000001
2016-01-14 15:48:32,308 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786498963_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:32,308 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786498963_0001_00_000001 completed successfully
2016-01-14 15:48:32,309 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:32,310 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786498963_0001_00_000003
2016-01-14 15:48:32,311 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:32,313 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:32,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:32,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786498963_0001_00_000003, launchTime=1452786512313
2016-01-14 15:48:32,315 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000003
2016-01-14 15:48:32,315 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786498963_0001_1_02_000000_0] to container: [container_1452786498963_0001_00_000003]
2016-01-14 15:48:32,315 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:32,316 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786498963_0001_00_000003 given task: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,316 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32 Starting to run new task attempt: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,316 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786498963_0001_1_02_000000_0] started. Is using containerId: [container_1452786498963_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 15:48:32,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0, startTime=1452786512316, containerId=container_1452786498963_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786498963_0001_00_000003/teamcity, completedLogs=
2016-01-14 15:48:32,316 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:32,317 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:32,316 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:32,317 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,317 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:32,318 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:32,318 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:32,318 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:32,318 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:32,319 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: A0C94E7A8AEA45F0A8819AD7A58541D3, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B71CA62645984090B1A2804D20C09E5D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:32,319 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:32,319 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,320 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:32,320 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,322 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786498963_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 15:48:32,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:32,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:32,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:32,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,325 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:32,325 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:32,325 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B71CA62645984090B1A2804D20C09E5D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:32,326 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:32,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:32,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B71CA62645984090B1A2804D20C09E5D
2016-01-14 15:48:32,326 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:32,327 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B71CA62645984090B1A2804D20C09E5D
2016-01-14 15:48:32,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:32,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,332 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:32,332 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:32,334 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,334 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,337 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A0C94E7A8AEA45F0A8819AD7A58541D3, ordinal: 2
2016-01-14 15:48:32,337 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0C94E7A8AEA45F0A8819AD7A58541D3, mem on start (mb), free: 254, total: 521, max: 672
2016-01-14 15:48:32,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 15:48:32,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:32,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:32,340 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:32,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B71CA62645984090B1A2804D20C09E5D
2016-01-14 15:48:32,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:32,340 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:32,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:32,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:32,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:32,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:32,341 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 15:48:32,342 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 15:48:32,342 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E2E2D7DC0F3041E7ABC17AC0FA602344, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:48:32,342 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B71CA62645984090B1A2804D20C09E5D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:32,342 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D262ACA9B2A94E9E8430EB2120B6B3A9, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:48:32,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:32,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E2E2D7DC0F3041E7ABC17AC0FA602344 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:32,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D262ACA9B2A94E9E8430EB2120B6B3A9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:32,346 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:32,346 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:32,347 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:32,347 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:32,347 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D262ACA9B2A94E9E8430EB2120B6B3A9 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:32,347 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:48:32,347 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:32,348 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E2E2D7DC0F3041E7ABC17AC0FA602344 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:32,348 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:48:32,348 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:32,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 15:48:32,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 15:48:32,348 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:32,349 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:32,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:32,354 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:32,354 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,354 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786498963_0001_1_01_000000_0_10016, runDuration: 1768849, ]
2016-01-14 15:48:32,355 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786498963_0001_1_00_000000_0_10014, runDuration: 13034431, ]
2016-01-14 15:48:32,356 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786498963_0001_1_00_000000_0_10014] to DISK_DIRECT, CompressedSize=8931, DecompressedSize=8927,EndTime=1452786512356, TimeTaken=1, Rate=8.52 MB/s
2016-01-14 15:48:32,356 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,357 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s)
2016-01-14 15:48:32,358 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E2E2D7DC0F3041E7ABC17AC0FA602344] #3 in 3ms
2016-01-14 15:48:32,359 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E2E2D7DC0F3041E7ABC17AC0FA602344] #3, status:false, isInterrupted:false
2016-01-14 15:48:32,361 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2016-01-14 15:48:32,361 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:32,362 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786498963_0001_1_01_000000_0_10016] to DISK_DIRECT, CompressedSize=142, DecompressedSize=138,EndTime=1452786512362, TimeTaken=4, Rate=0.03 MB/s
2016-01-14 15:48:32,363 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,362 WARN util.TupleHasher (TupleHasher.java:merge(73)) - two unequal Hasher instances for the same key field position found: cascading.platform.hadoop.TestLongComparator@42d1ff2e, and: cascading.platform.hadoop.TestLongComparator@68c84747
2016-01-14 15:48:32,364 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] streamed: true, id: E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:32,364 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"], id: B71CA62645984090B1A2804D20C09E5D
2016-01-14 15:48:32,364 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:32,365 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D262ACA9B2A94E9E8430EB2120B6B3A9] #4 in 7ms
2016-01-14 15:48:32,363 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 8931 bytes from disk
2016-01-14 15:48:32,365 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:32,365 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:32,364 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"]
2016-01-14 15:48:32,366 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D262ACA9B2A94E9E8430EB2120B6B3A9] #4, status:false, isInterrupted:false
2016-01-14 15:48:32,366 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] E7AFE3E8781F465AA8FDCFF0ADECFA06, for 2 inputs
2016-01-14 15:48:32,367 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 8918 bytes
2016-01-14 15:48:32,367 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,366 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:32,367 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] E7AFE3E8781F465AA8FDCFF0ADECFA06, for 2 inputs
2016-01-14 15:48:32,368 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:32,369 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 142 bytes from disk
2016-01-14 15:48:32,369 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:32,369 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:32,369 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:48:32,370 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:48:32,370 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 129 bytes
2016-01-14 15:48:32,370 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,371 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:32,371 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A0C94E7A8AEA45F0A8819AD7A58541D3, all 2 inputs ready in: 00:00:00.000
2016-01-14 15:48:32,374 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7737d902; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@73884f73
2016-01-14 15:48:32,376 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@27e675fc; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@b4c07db
2016-01-14 15:48:32,399 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:32,400 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:32,400 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,401 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786498963_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 15:48:32,402 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,402 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0C94E7A8AEA45F0A8819AD7A58541D3, mem on close (mb), free: 244, total: 521, max: 672
2016-01-14 15:48:32,402 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,403 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:32,403 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:32,403 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:32,404 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:32,404 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:32,404 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,405 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:32,406 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:32,406 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:32,406 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:32,406 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:32 Completed running task attempt: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,407 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:32,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786512372, Process_Duration=30, Process_End_Time=1452786512402, Read_Duration=7, Tuples_Read=124, Tuples_Written=199, Write_Duration=2][cascading.flow.StepCounters Tuples_Written=199]]
2016-01-14 15:48:32,409 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:32,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:32,410 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,410 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000003
2016-01-14 15:48:32,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0, startTime=1452786512316, finishTime=1452786512408, timeTaken=92, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512372, Process_Duration=30, Process_End_Time=1452786512402, Read_Duration=7, Tuples_Read=124, Tuples_Written=199, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=199
2016-01-14 15:48:32,411 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:32,411 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:32,411 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,412 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:32,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_FINISHED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, taskId=task_1452786498963_0001_1_02_000000, startTime=1452786512316, finishTime=1452786512411, timeTaken=95, status=SUCCEEDED, successfulAttemptID=attempt_1452786498963_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512372, Process_Duration=30, Process_End_Time=1452786512402, Read_Duration=7, Tuples_Read=124, Tuples_Written=199, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=199
2016-01-14 15:48:32,413 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:32,413 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] : 1
2016-01-14 15:48:32,413 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:32,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:32,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_FINISHED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, vertexId=vertex_1452786498963_0001_1_02, initRequestedTime=1452786499174, initedTime=1452786499200, startRequestedTime=1452786499222, startedTime=1452786499222, finishTime=1452786512413, timeTaken=13191, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512372, Process_Duration=30, Process_End_Time=1452786512402, Read_Duration=7, Tuples_Read=124, Tuples_Written=199, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=199, vertexStats=firstTaskStartTime=1452786512316, firstTasksToStart=[ task_1452786498963_0001_1_02_000000 ], lastTaskFinishTime=1452786512408, lastTasksToFinish=[ task_1452786498963_0001_1_02_000000 ], minTaskDuration=92, maxTaskDuration=92, avgTaskDuration=92.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786498963_0001_1_02_000000 ], longestDurationTasks=[ task_1452786498963_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:32,415 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:32,415 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:32,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786498963_0001_00_000003, stoppedTime=1452786512416, exitStatus=0
2016-01-14 15:48:32,416 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:32,416 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:32,416 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786498963_0001_1
2016-01-14 15:48:32,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786498963_0001_1
2016-01-14 15:48:32,416 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B71CA62645984090B1A2804D20C09E5D for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:32,418 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:32,421 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786498963_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 15:48:32,421 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000003: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:32,421 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786498963_0001_00_000003, Exiting
2016-01-14 15:48:32,421 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786498963_0001_00_000003
2016-01-14 15:48:32,421 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786498963_0001_00_000003
2016-01-14 15:48:32,423 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786498963_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:32,423 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786498963_0001_00_000003 completed successfully
2016-01-14 15:48:32,428 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:32,428 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:32,431 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786498963_0001_1, queueSize=0
2016-01-14 15:48:32,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:32,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786498963_0001_1, startTime=1452786499168, finishTime=1452786512428, timeTaken=13260, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=83619, FILE_BYTES_WRITTEN=25103, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=440, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=927, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1639448576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=419, OUTPUT_BYTES=9163, OUTPUT_BYTES_WITH_OVERHEAD=9065, OUTPUT_BYTES_PHYSICAL=9073, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358359511547, Process_Duration=61, Process_End_Time=4358359511608, Read_Duration=10, Tuples_Read=344, Tuples_Written=419, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=220, Tuples_Written=199
2016-01-14 15:48:32,434 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786498963_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:32,434 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786498963_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:32,434 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786498963_0001_00_000003
2016-01-14 15:48:32,435 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:32 Completed Dag: dag_1452786498963_0001_1
2016-01-14 15:48:32,437 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786498963_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:32,438 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786498963_0001_1_post
2016-01-14 15:48:32,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,438 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:32,440 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1452786498963_0001
2016-01-14 15:48:32,443 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:32,443 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:32,443 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:32,444 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:32,444 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:32,444 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:32,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:32,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:32,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:32,445 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:32,446 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:32,446 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:32,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:32,452 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001
2016-01-14 15:48:32,452 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] completed in: 00:13.510
2016-01-14 15:48:32,501 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward
2016-01-14 15:48:32,523 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:32,523 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:32,554 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-01-14 15:48:32,557 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.012
2016-01-14 15:48:32,558 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 15:48:32,558 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [apache] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 15:48:32,579 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:32,579 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:32,618 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 15:48:32,632 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-01-14 15:48:32,633 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:32,633 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:32,633 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:32,646 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:32,656 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false
2016-01-14 15:48:32,692 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:32,693 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:32,693 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"]
2016-01-14 15:48:32,693 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:32,693 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:32,693 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:32,694 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ve/false_null_false_false
2016-01-14 15:48:32,694 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:32,704 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:32,709 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:32,709 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:32,709 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:32,719 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001 doesn't exist and is created
2016-01-14 15:48:32,737 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:32,738 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:32,738 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001
2016-01-14 15:48:32,742 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786512709_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:32,750 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:32,751 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:32,752 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:32,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:32,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001/localmode-log-dir
2016-01-14 15:48:32,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001/localmode-log-dir/history.txt.appattempt_1452786512709_0001_000000, maxErrors=10
2016-01-14 15:48:32,753 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:32,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786512709_0001_000000, appSubmitTime=1452786512741, launchTime=1452786512741
2016-01-14 15:48:32,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:32,760 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:32,761 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:32,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:32,766 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39962
2016-01-14 15:48:32,768 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:32,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786512709_0001_000000, startTime=1452786512771
2016-01-14 15:48:32,771 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:32,771 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:32,838 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:32,838 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:32,838 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1452786512709_0001, dagName=[C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-01-14 15:48:32,852 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-01-14 15:48:32,854 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786512709_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001/localmode-log-dir/dag_1452786512709_0001_1.dot
2016-01-14 15:48:32,855 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001/localmode-log-dir/dag_1452786512709_0001_1-tez-dag.pb.txt
2016-01-14 15:48:32,880 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786512709_0001_1
2016-01-14 15:48:32,880 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,880 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-01-14 15:48:32 Running Dag: dag_1452786512709_0001_1
2016-01-14 15:48:32,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786512709_0001_1, submitTime=1452786512852
2016-01-14 15:48:32,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:32,896 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,896 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:32,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786512709_0001_1, initTime=1452786512895
2016-01-14 15:48:32,898 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786512709_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:32,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:32,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_STARTED]: dagID=dag_1452786512709_0001_1, startTime=1452786512898
2016-01-14 15:48:32,898 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:32,899 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786512709_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:32,899 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FF7D39CC29834167BCDC47B8A32E1623 : {0C68B8C0E004462B9AE58835AEF89075={InputName=0C68B8C0E004462B9AE58835AEF89075}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:32,899 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0C68B8C0E004462B9AE58835AEF89075, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:32,899 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,900 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,900 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,900 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1452786512709_0001, dagName=[C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-01-14 15:48:32,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:32,900 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]. Starting root input initializers: 1
2016-01-14 15:48:32,901 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:32,901 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,903 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0C68B8C0E004462B9AE58835AEF89075 on vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,904 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:32,904 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,904 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,905 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,905 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=027E5FADB04D43FF8048A32FD649448C, vertexId=vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:32,905 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=027E5FADB04D43FF8048A32FD649448C, vertexId=vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,908 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:32,908 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0C68B8C0E004462B9AE58835AEF89075 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:32,909 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=027E5FADB04D43FF8048A32FD649448C, vertexName=F60F05D3CB2842539B07783029236D31, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:32,910 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:32,910 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:32,912 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=027E5FADB04D43FF8048A32FD649448C, vertexId=vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,913 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:32,913 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F60F05D3CB2842539B07783029236D31, vertexId=vertex_1452786512709_0001_1_01, initRequestedTime=1452786512901, initedTime=1452786512913, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:32,914 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:32,914 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:32,915 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0C68B8C0E004462B9AE58835AEF89075 on vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,915 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:32,915 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0C68B8C0E004462B9AE58835AEF89075=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:32,915 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] parallelism set to 1
2016-01-14 15:48:32,916 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:32,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786512709_0001_1_00, eventCount=1
2016-01-14 15:48:32,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, vertexId=vertex_1452786512709_0001_1_00, initRequestedTime=1452786512899, initedTime=1452786512916, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:32,916 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] with distanceFromRoot: 0
2016-01-14 15:48:32,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:32,916 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:32,917 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:32,917 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786512709_0001_1_00, startRequestedTime=1452786512917, startedTime=1452786512917
2016-01-14 15:48:32,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:32,917 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:32,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:32,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_STARTED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, taskId=task_1452786512709_0001_1_00_000000, scheduledTime=1452786512918, launchTime=1452786512918
2016-01-14 15:48:32,918 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:32,918 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786512709_0001_1_00 for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] numStartedSources: 1 numSources: 1
2016-01-14 15:48:32,918 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] with distanceFromRoot: 1
2016-01-14 15:48:32,918 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786512709_0001, with dag id: dag_1452786512709_0001_1
2016-01-14 15:48:32,919 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,919 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FF7D39CC29834167BCDC47B8A32E1623 in vertex: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,920 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F60F05D3CB2842539B07783029236D31 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:32,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786512709_0001_1_01, startRequestedTime=1452786512919, startedTime=1452786512919
2016-01-14 15:48:32,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:32,920 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:32,920 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: FF7D39CC29834167BCDC47B8A32E1623, VertexParallelism: 1, TaskAttemptID:attempt_1452786512709_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C68B8C0E004462B9AE58835AEF89075, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F60F05D3CB2842539B07783029236D31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:32,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:32,921 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:32,924 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:32,924 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:32,925 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:32,925 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786512709_0001_00_000001
2016-01-14 15:48:32,926 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:32,928 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786512709_0001_00_000001
2016-01-14 15:48:32,928 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:32,929 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:32,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:32,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786512709_0001_00_000001, launchTime=1452786512929
2016-01-14 15:48:32,931 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786512709_0001_00_000001
2016-01-14 15:48:32,932 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786512709_0001_1_00_000000_0] to container: [container_1452786512709_0001_00_000001]
2016-01-14 15:48:32,932 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:32,932 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786512709_0001_00_000001 given task: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,932 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786512709_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,932 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786512709_0001_1_00_000000_0] started. Is using containerId: [container_1452786512709_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:32,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0, startTime=1452786512932, containerId=container_1452786512709_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786512709_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:32,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:32,933 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:32,933 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:32 Starting to run new task attempt: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,934 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,934 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,935 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:32,935 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:32,935 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:32,935 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:32,936 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:32,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: FF7D39CC29834167BCDC47B8A32E1623, VertexParallelism: 1, TaskAttemptID:attempt_1452786512709_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C68B8C0E004462B9AE58835AEF89075, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F60F05D3CB2842539B07783029236D31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:32,936 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:32,937 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:32,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0C68B8C0E004462B9AE58835AEF89075, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:32,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F60F05D3CB2842539B07783029236D31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:32,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:32,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:32,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,940 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,941 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:32,942 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786512709_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:32,942 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:32,943 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,946 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:32,946 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,950 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF7D39CC29834167BCDC47B8A32E1623, ordinal: 0
2016-01-14 15:48:32,950 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF7D39CC29834167BCDC47B8A32E1623, mem on start (mb), free: 126, total: 521, max: 672
2016-01-14 15:48:32,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:32,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:32,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:32,951 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:32,951 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:32,952 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:32,952 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:32,952 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:32,952 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:32,953 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0C68B8C0E004462B9AE58835AEF89075, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:32,953 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F60F05D3CB2842539B07783029236D31, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:32,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:32,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0C68B8C0E004462B9AE58835AEF89075 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:32,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:32,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:32,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,955 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:32,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:32,956 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,956 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:32,957 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2406cff2
2016-01-14 15:48:32,958 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2406cff2
2016-01-14 15:48:32,958 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:32,958 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:32,959 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:32,961 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,961 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: AF08A6C1A8C44B36BEFE05E64CC58DCE
2016-01-14 15:48:32,962 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] AF08A6C1A8C44B36BEFE05E64CC58DCE
2016-01-14 15:48:32,963 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@263bff7b; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2d01f7d9; comparator=cascading.tuple.hadoop.util.TupleComparator@5c36dea4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:32,963 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:32,963 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:32,963 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:32,983 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:32,984 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:32,984 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:32,984 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:32,984 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:32,985 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:32,985 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:32,986 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF7D39CC29834167BCDC47B8A32E1623, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:32,991 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,991 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF7D39CC29834167BCDC47B8A32E1623, mem on close (mb), free: 187, total: 516, max: 672
2016-01-14 15:48:32,991 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:32,992 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:32,992 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:32,992 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:32,996 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:32 Completed running task attempt: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:33,000 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:33,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786512709_0001_1_00, eventCount=1
2016-01-14 15:48:33,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:33,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0, startTime=1452786512932, finishTime=1452786513001, timeTaken=69, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512986, Process_Duration=5, Process_End_Time=1452786512991, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:33,001 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:33,002 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:33,002 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:33,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:33,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_FINISHED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, taskId=task_1452786512709_0001_1_00_000000, startTime=1452786512932, finishTime=1452786513002, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1452786512709_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512986, Process_Duration=5, Process_End_Time=1452786512991, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:33,003 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:33,003 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] : 1
2016-01-14 15:48:33,003 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:33,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786512986, Process_Duration=5, Process_End_Time=1452786512991, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:33,004 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:33,004 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,004 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786512709_0001_00_000001
2016-01-14 15:48:33,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:33,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_FINISHED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, vertexId=vertex_1452786512709_0001_1_00, initRequestedTime=1452786512899, initedTime=1452786512916, startRequestedTime=1452786512917, startedTime=1452786512917, finishTime=1452786513003, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512986, Process_Duration=5, Process_End_Time=1452786512991, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786512932, firstTasksToStart=[ task_1452786512709_0001_1_00_000000 ], lastTaskFinishTime=1452786513001, lastTasksToFinish=[ task_1452786512709_0001_1_00_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786512709_0001_1_00_000000 ], longestDurationTasks=[ task_1452786512709_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:33,006 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:33,007 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:33,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786512709_0001_00_000001, stoppedTime=1452786513007, exitStatus=0
2016-01-14 15:48:33,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:33,007 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:33,008 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] attempt: attempt_1452786512709_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:33,008 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:33,009 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:33,009 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:33,009 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786512709_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:33,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_STARTED]: vertexName=F60F05D3CB2842539B07783029236D31, taskId=task_1452786512709_0001_1_01_000000, scheduledTime=1452786513010, launchTime=1452786513010
2016-01-14 15:48:33,010 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:33,010 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:33,010 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:33,011 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:33,011 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: F60F05D3CB2842539B07783029236D31, VertexParallelism: 1, TaskAttemptID:attempt_1452786512709_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF7D39CC29834167BCDC47B8A32E1623, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=027E5FADB04D43FF8048A32FD649448C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:33,011 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:33,012 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:33,012 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786512709_0001_00_000002
2016-01-14 15:48:33,013 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:33,014 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786512709_0001_00_000002
2016-01-14 15:48:33,014 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:33,015 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:33,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:33,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786512709_0001_00_000002, launchTime=1452786513015
2016-01-14 15:48:33,015 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786512709_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:33,016 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786512709_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:33,016 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786512709_0001_00_000002
2016-01-14 15:48:33,017 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786512709_0001_00_000001, Exiting
2016-01-14 15:48:33,017 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786512709_0001_00_000001
2016-01-14 15:48:33,017 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786512709_0001_00_000001
2016-01-14 15:48:33,017 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786512709_0001_1_01_000000_0] to container: [container_1452786512709_0001_00_000002]
2016-01-14 15:48:33,020 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:33,020 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786512709_0001_00_000002 given task: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,021 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786512709_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,021 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786512709_0001_1_01_000000_0] started. Is using containerId: [container_1452786512709_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:33,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:33 Starting to run new task attempt: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,021 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,021 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,021 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:33,022 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:33,022 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:33,022 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:33,022 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:33,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: F60F05D3CB2842539B07783029236D31, VertexParallelism: 1, TaskAttemptID:attempt_1452786512709_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF7D39CC29834167BCDC47B8A32E1623, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=027E5FADB04D43FF8048A32FD649448C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:33,022 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:33,021 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786512709_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:33,028 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786512709_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:33,026 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:33,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F60F05D3CB2842539B07783029236D31, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0, startTime=1452786513020, containerId=container_1452786512709_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786512709_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:33,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,028 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786512709_0001_00_000001 completed successfully
2016-01-14 15:48:33,029 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:33,030 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:33,030 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:33,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=027E5FADB04D43FF8048A32FD649448C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:33,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:33,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 027E5FADB04D43FF8048A32FD649448C
2016-01-14 15:48:33,031 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:33,031 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 027E5FADB04D43FF8048A32FD649448C
2016-01-14 15:48:33,031 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:33,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FF7D39CC29834167BCDC47B8A32E1623, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:33,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:33,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,037 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:33,037 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:33,038 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,041 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F60F05D3CB2842539B07783029236D31, ordinal: 1
2016-01-14 15:48:33,041 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F60F05D3CB2842539B07783029236D31, mem on start (mb), free: 179, total: 516, max: 672
2016-01-14 15:48:33,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:33,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:33,044 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:33,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 027E5FADB04D43FF8048A32FD649448C
2016-01-14 15:48:33,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:33,045 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:33,045 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:33,045 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:33,045 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:33,046 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:33,046 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:33,046 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 027E5FADB04D43FF8048A32FD649448C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:33,046 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FF7D39CC29834167BCDC47B8A32E1623, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:33,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:33,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,047 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:33,048 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:33,048 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FF7D39CC29834167BCDC47B8A32E1623 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:33,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FF7D39CC29834167BCDC47B8A32E1623 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:33,048 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:33,049 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:33,049 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:33,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:33,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:33,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:33,050 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,056 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:33,056 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786512709_0001_1_00_000000_0_10022, runDuration: 58793, ]
2016-01-14 15:48:33,056 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: AF08A6C1A8C44B36BEFE05E64CC58DCE
2016-01-14 15:48:33,056 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"], id: 027E5FADB04D43FF8048A32FD649448C
2016-01-14 15:48:33,057 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"]
2016-01-14 15:48:33,057 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] AF08A6C1A8C44B36BEFE05E64CC58DCE, for 1 inputs
2016-01-14 15:48:33,058 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786512709_0001_1_00_000000_0_10022] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786513058, TimeTaken=2, Rate=0.28 MB/s
2016-01-14 15:48:33,058 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,058 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:33,058 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FF7D39CC29834167BCDC47B8A32E1623] #5 in 2ms
2016-01-14 15:48:33,059 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FF7D39CC29834167BCDC47B8A32E1623] #5, status:false, isInterrupted:false
2016-01-14 15:48:33,060 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:33,060 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:33,060 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:33,060 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:33,061 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:33,061 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,061 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F60F05D3CB2842539B07783029236D31, all 1 inputs ready in: 00:00:00.004
2016-01-14 15:48:33,062 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:33,062 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@56a6699e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@47603083
2016-01-14 15:48:33,069 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:33,069 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:33,069 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,070 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786512709_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:33,070 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,070 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F60F05D3CB2842539B07783029236D31, mem on close (mb), free: 175, total: 516, max: 672
2016-01-14 15:48:33,071 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,071 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:33,071 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:33,071 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:33,072 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:33,072 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:33 Completed running task attempt: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,073 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:33,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F60F05D3CB2842539B07783029236D31, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0, startTime=1452786513020, finishTime=1452786513073, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513062, Process_Duration=8, Process_End_Time=1452786513070, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:33,074 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:33,074 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_FINISHED]: vertexName=F60F05D3CB2842539B07783029236D31, taskId=task_1452786512709_0001_1_01_000000, startTime=1452786513020, finishTime=1452786513074, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452786512709_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513062, Process_Duration=8, Process_End_Time=1452786513070, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:33,075 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:33,075 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:33,075 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] : 1
2016-01-14 15:48:33,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:33,075 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:33,076 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:33,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_FINISHED]: vertexName=F60F05D3CB2842539B07783029236D31, vertexId=vertex_1452786512709_0001_1_01, initRequestedTime=1452786512901, initedTime=1452786512913, startRequestedTime=1452786512919, startedTime=1452786512919, finishTime=1452786513075, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513062, Process_Duration=8, Process_End_Time=1452786513070, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786513020, firstTasksToStart=[ task_1452786512709_0001_1_01_000000 ], lastTaskFinishTime=1452786513073, lastTasksToFinish=[ task_1452786512709_0001_1_01_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786512709_0001_1_01_000000 ], longestDurationTasks=[ task_1452786512709_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:33,077 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:33,077 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:33,077 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:33,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786512709_0001_00_000002, stoppedTime=1452786513077, exitStatus=0
2016-01-14 15:48:33,077 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:33,078 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:33,078 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786512709_0001_1
2016-01-14 15:48:33,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786512709_0001_1
2016-01-14 15:48:33,078 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:33,078 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 027E5FADB04D43FF8048A32FD649448C for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:33,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:33,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786513062, Process_Duration=8, Process_End_Time=1452786513070, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:33,080 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:33,080 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,081 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786512709_0001_00_000002
2016-01-14 15:48:33,081 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786512709_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:33,081 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786512709_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:33,082 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786512709_0001_00_000002, Exiting
2016-01-14 15:48:33,082 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786512709_0001_00_000002
2016-01-14 15:48:33,082 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786512709_0001_00_000002
2016-01-14 15:48:33,082 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786512709_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:33,082 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786512709_0001_00_000002 completed successfully
2016-01-14 15:48:33,090 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786512709_0001_1, queueSize=0
2016-01-14 15:48:33,090 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:33,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786512709_0001_1, startTime=1452786512898, finishTime=1452786513088, timeTaken=190, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=859, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1083179008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573026048, Process_Duration=13, Process_End_Time=2905573026061, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:33,091 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786512709_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:33,091 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786512709_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:33,092 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786512709_0001_00_000002
2016-01-14 15:48:33,092 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:33 Completed Dag: dag_1452786512709_0001_1
2016-01-14 15:48:33,095 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786512709_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:33,095 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786512709_0001_1_post
2016-01-14 15:48:33,096 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,097 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:33,097 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1452786512709_0001
2016-01-14 15:48:33,100 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:33,101 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:33,101 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:33,101 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:33,101 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:33,101 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:33,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:33,103 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:33,103 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:33,103 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:33,104 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:33,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:33,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:33,110 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001
2016-01-14 15:48:33,111 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.418
2016-01-14 15:48:33,165 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:33,166 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:33,204 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:48:33,214 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-01-14 15:48:33,215 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:33,215 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:33,215 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:33,230 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:33,239 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false
2016-01-14 15:48:33,278 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:33,278 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:33,278 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"]
2016-01-14 15:48:33,278 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:33,278 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:33,279 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:33,280 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/true_null_false_false
2016-01-14 15:48:33,280 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:33,289 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:33,294 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:33,294 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:33,294 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:33,304 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001 doesn't exist and is created
2016-01-14 15:48:33,321 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:33,321 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:33,321 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001
2016-01-14 15:48:33,325 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786513294_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:33,333 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:33,334 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:33,335 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:33,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:33,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001/localmode-log-dir
2016-01-14 15:48:33,336 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001/localmode-log-dir/history.txt.appattempt_1452786513294_0001_000000, maxErrors=10
2016-01-14 15:48:33,336 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:33,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786513294_0001_000000, appSubmitTime=1452786513324, launchTime=1452786513325
2016-01-14 15:48:33,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:33,344 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:33,346 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:42363
2016-01-14 15:48:33,347 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:33,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:33,350 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:33,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786513294_0001_000000, startTime=1452786513353
2016-01-14 15:48:33,353 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:33,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:33,421 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:33,421 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:33,421 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1452786513294_0001, dagName=[B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-01-14 15:48:33,435 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-01-14 15:48:33,436 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786513294_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001/localmode-log-dir/dag_1452786513294_0001_1.dot
2016-01-14 15:48:33,438 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001/localmode-log-dir/dag_1452786513294_0001_1-tez-dag.pb.txt
2016-01-14 15:48:33,461 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786513294_0001_1
2016-01-14 15:48:33,462 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,462 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-01-14 15:48:33 Running Dag: dag_1452786513294_0001_1
2016-01-14 15:48:33,475 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786513294_0001_1, submitTime=1452786513435
2016-01-14 15:48:33,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:33,476 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,477 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:33,478 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786513294_0001_1, initTime=1452786513475
2016-01-14 15:48:33,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:33,478 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786513294_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:33,479 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_STARTED]: dagID=dag_1452786513294_0001_1, startTime=1452786513479
2016-01-14 15:48:33,479 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:33,480 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786513294_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:33,480 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F9FC9A7F42574E3F942ED332EBFA8AB2 : {849A9EB1F5744007A9A5F57814780DE9={InputName=849A9EB1F5744007A9A5F57814780DE9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:33,480 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 849A9EB1F5744007A9A5F57814780DE9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:33,480 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,480 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,480 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,479 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:33,481 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]. Starting root input initializers: 1
2016-01-14 15:48:33,482 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1452786513294_0001, dagName=[B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-01-14 15:48:33,483 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:33,483 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 849A9EB1F5744007A9A5F57814780DE9 on vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,483 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,486 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:33,486 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,487 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,487 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,487 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8199A0D1EEF1492B8D41644EB11897FF, vertexId=vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:33,487 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8199A0D1EEF1492B8D41644EB11897FF, vertexId=vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,486 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:33,488 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 849A9EB1F5744007A9A5F57814780DE9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:33,489 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:33,489 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:33,490 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8199A0D1EEF1492B8D41644EB11897FF, vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:33,490 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8199A0D1EEF1492B8D41644EB11897FF, vertexId=vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:33,492 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:33,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, vertexId=vertex_1452786513294_0001_1_01, initRequestedTime=1452786513483, initedTime=1452786513492, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:33,493 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:33,493 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 849A9EB1F5744007A9A5F57814780DE9 on vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,493 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:33,494 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {849A9EB1F5744007A9A5F57814780DE9=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:33,494 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] parallelism set to 1
2016-01-14 15:48:33,494 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:33,494 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786513294_0001, with dag id: dag_1452786513294_0001_1
2016-01-14 15:48:33,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786513294_0001_1_00, eventCount=1
2016-01-14 15:48:33,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:33,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, vertexId=vertex_1452786513294_0001_1_00, initRequestedTime=1452786513480, initedTime=1452786513498, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:33,498 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] with distanceFromRoot: 0
2016-01-14 15:48:33,498 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:33,499 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:33,499 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:33,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786513294_0001_1_00, startRequestedTime=1452786513499, startedTime=1452786513499
2016-01-14 15:48:33,501 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:33,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:33,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_STARTED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, taskId=task_1452786513294_0001_1_00_000000, scheduledTime=1452786513501, launchTime=1452786513501
2016-01-14 15:48:33,502 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:33,502 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786513294_0001_1_00 for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] numStartedSources: 1 numSources: 1
2016-01-14 15:48:33,502 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] with distanceFromRoot: 1
2016-01-14 15:48:33,503 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,503 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F9FC9A7F42574E3F942ED332EBFA8AB2 in vertex: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,503 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DDD4DA62B63D4F7EB1BA28AAD2B36F10 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:33,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:33,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786513294_0001_1_01, startRequestedTime=1452786513503, startedTime=1452786513503
2016-01-14 15:48:33,504 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:33,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: F9FC9A7F42574E3F942ED332EBFA8AB2, VertexParallelism: 1, TaskAttemptID:attempt_1452786513294_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=849A9EB1F5744007A9A5F57814780DE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:33,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:33,505 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:33,506 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:33,506 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:33,507 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:33,508 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786513294_0001_00_000001
2016-01-14 15:48:33,508 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:33,510 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786513294_0001_00_000001
2016-01-14 15:48:33,510 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:33,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:33,511 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:33,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786513294_0001_00_000001, launchTime=1452786513510
2016-01-14 15:48:33,511 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786513294_0001_00_000001
2016-01-14 15:48:33,512 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786513294_0001_1_00_000000_0] to container: [container_1452786513294_0001_00_000001]
2016-01-14 15:48:33,512 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:33,513 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786513294_0001_00_000001 given task: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,513 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786513294_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,513 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786513294_0001_1_00_000000_0] started. Is using containerId: [container_1452786513294_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:33 Starting to run new task attempt: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,514 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:33,514 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,514 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,515 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:33,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0, startTime=1452786513513, containerId=container_1452786513294_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786513294_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:33,515 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:33,515 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:33,516 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:33,516 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:33,517 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:33,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: F9FC9A7F42574E3F942ED332EBFA8AB2, VertexParallelism: 1, TaskAttemptID:attempt_1452786513294_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=849A9EB1F5744007A9A5F57814780DE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:33,517 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:33,515 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:33,518 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:33,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=849A9EB1F5744007A9A5F57814780DE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:33,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,525 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786513294_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:33,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:33,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,526 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:33,528 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:33,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:33,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,533 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:33,533 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,534 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:33,535 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,541 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9FC9A7F42574E3F942ED332EBFA8AB2, ordinal: 0
2016-01-14 15:48:33,541 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FC9A7F42574E3F942ED332EBFA8AB2, mem on start (mb), free: 85, total: 516, max: 672
2016-01-14 15:48:33,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:33,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:33,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:33,542 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:33,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:33,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:33,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:33,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:33,544 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:33,544 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 849A9EB1F5744007A9A5F57814780DE9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:33,544 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DDD4DA62B63D4F7EB1BA28AAD2B36F10, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:33,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:33,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 849A9EB1F5744007A9A5F57814780DE9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:33,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,546 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:33,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:33,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:33,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:33,547 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,547 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:33,548 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@66e2cf6e
2016-01-14 15:48:33,549 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@66e2cf6e
2016-01-14 15:48:33,549 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:33,549 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:33,551 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:33,552 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,552 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 2C49410EB8674D45AF855319EC703DD2
2016-01-14 15:48:33,553 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 2C49410EB8674D45AF855319EC703DD2
2016-01-14 15:48:33,555 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@30bf57bb; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6218cefd; comparator=cascading.tuple.hadoop.util.TupleComparator@4abd765b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:33,555 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:33,555 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:33,556 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:33,584 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:33,585 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:33,585 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:33,585 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:33,586 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:33,586 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:33,587 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:33,587 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9FC9A7F42574E3F942ED332EBFA8AB2, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:33,592 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,592 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FC9A7F42574E3F942ED332EBFA8AB2, mem on close (mb), free: 190, total: 519, max: 672
2016-01-14 15:48:33,593 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:33,593 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:33,593 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:33,593 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:35,220 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:35 Completed running task attempt: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:35,224 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:35,224 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786513294_0001_1_00, eventCount=1
2016-01-14 15:48:35,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1141, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786513588, Process_Duration=4, Process_End_Time=1452786513592, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:35,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0, startTime=1452786513513, finishTime=1452786515225, timeTaken=1712, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1141, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513588, Process_Duration=4, Process_End_Time=1452786513592, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:35,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:35,225 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:35,226 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,226 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786513294_0001_00_000001
2016-01-14 15:48:35,225 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:35,227 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:35,227 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:35,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:35,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_FINISHED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, taskId=task_1452786513294_0001_1_00_000000, startTime=1452786513513, finishTime=1452786515228, timeTaken=1715, status=SUCCEEDED, successfulAttemptID=attempt_1452786513294_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1141, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513588, Process_Duration=4, Process_End_Time=1452786513592, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:35,229 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:35,229 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] : 1
2016-01-14 15:48:35,229 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:35,230 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:35,230 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_FINISHED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, vertexId=vertex_1452786513294_0001_1_00, initRequestedTime=1452786513480, initedTime=1452786513498, startRequestedTime=1452786513499, startedTime=1452786513499, finishTime=1452786515229, timeTaken=1730, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1141, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513588, Process_Duration=4, Process_End_Time=1452786513592, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786513513, firstTasksToStart=[ task_1452786513294_0001_1_00_000000 ], lastTaskFinishTime=1452786515225, lastTasksToFinish=[ task_1452786513294_0001_1_00_000000 ], minTaskDuration=1712, maxTaskDuration=1712, avgTaskDuration=1712.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786513294_0001_1_00_000000 ], longestDurationTasks=[ task_1452786513294_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:35,230 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:35,231 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:35,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786513294_0001_00_000001, stoppedTime=1452786515231, exitStatus=0
2016-01-14 15:48:35,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:35,231 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:35,232 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] attempt: attempt_1452786513294_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:35,232 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:35,232 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:35,233 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:35,233 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786513294_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:35,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_STARTED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, taskId=task_1452786513294_0001_1_01_000000, scheduledTime=1452786515233, launchTime=1452786515233
2016-01-14 15:48:35,233 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:35,234 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:35,234 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:35,234 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: DDD4DA62B63D4F7EB1BA28AAD2B36F10, VertexParallelism: 1, TaskAttemptID:attempt_1452786513294_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8199A0D1EEF1492B8D41644EB11897FF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:35,235 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:35,235 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:35,233 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:35,236 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786513294_0001_00_000002
2016-01-14 15:48:35,236 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:35,238 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786513294_0001_00_000002
2016-01-14 15:48:35,238 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:35,239 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786513294_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:35,239 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786513294_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:35,239 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:35,239 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786513294_0001_00_000001, Exiting
2016-01-14 15:48:35,239 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786513294_0001_00_000001
2016-01-14 15:48:35,240 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786513294_0001_00_000001
2016-01-14 15:48:35,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:35,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786513294_0001_00_000002, launchTime=1452786515239
2016-01-14 15:48:35,241 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786513294_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:35,242 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786513294_0001_00_000001 completed successfully
2016-01-14 15:48:35,242 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:35,242 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786513294_0001_00_000002
2016-01-14 15:48:35,243 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786513294_0001_1_01_000000_0] to container: [container_1452786513294_0001_00_000002]
2016-01-14 15:48:35,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:35,243 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786513294_0001_00_000002 given task: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,243 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786513294_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,244 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786513294_0001_1_01_000000_0] started. Is using containerId: [container_1452786513294_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:35 Starting to run new task attempt: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,244 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,244 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,244 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:35,244 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:35,244 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:35,245 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:35,245 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:35,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:35,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: DDD4DA62B63D4F7EB1BA28AAD2B36F10, VertexParallelism: 1, TaskAttemptID:attempt_1452786513294_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8199A0D1EEF1492B8D41644EB11897FF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:35,245 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:35,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0, startTime=1452786515243, containerId=container_1452786513294_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786513294_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:35,246 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:35,246 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:35,247 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:35,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,248 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786513294_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:35,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:35,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8199A0D1EEF1492B8D41644EB11897FF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:35,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:35,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:35,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8199A0D1EEF1492B8D41644EB11897FF
2016-01-14 15:48:35,251 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:35,251 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8199A0D1EEF1492B8D41644EB11897FF
2016-01-14 15:48:35,253 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:35,253 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:35,253 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,253 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:35,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,264 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DDD4DA62B63D4F7EB1BA28AAD2B36F10, ordinal: 1
2016-01-14 15:48:35,264 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:35,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8199A0D1EEF1492B8D41644EB11897FF
2016-01-14 15:48:35,264 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DDD4DA62B63D4F7EB1BA28AAD2B36F10, mem on start (mb), free: 87, total: 428, max: 672
2016-01-14 15:48:35,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:35,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:35,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:35,266 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:35,266 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:35,266 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:35,266 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:35,267 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:35,267 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:35,267 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8199A0D1EEF1492B8D41644EB11897FF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:35,268 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F9FC9A7F42574E3F942ED332EBFA8AB2, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:35,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:35,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,269 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:35,269 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:35,269 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F9FC9A7F42574E3F942ED332EBFA8AB2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:35,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F9FC9A7F42574E3F942ED332EBFA8AB2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:35,269 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:35,270 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:35,270 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:35,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:35,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:35,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:35,273 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,274 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786513294_0001_1_00_000000_0_10029, runDuration: 1693322, ]
2016-01-14 15:48:35,275 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786513294_0001_1_00_000000_0_10029] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786515275, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:35,275 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,275 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:35,275 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F9FC9A7F42574E3F942ED332EBFA8AB2] #6 in 1ms
2016-01-14 15:48:35,276 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F9FC9A7F42574E3F942ED332EBFA8AB2] #6, status:false, isInterrupted:false
2016-01-14 15:48:35,277 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:35,278 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:35,278 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:35,278 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:35,278 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:35,279 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 2C49410EB8674D45AF855319EC703DD2
2016-01-14 15:48:35,279 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"], id: 8199A0D1EEF1492B8D41644EB11897FF
2016-01-14 15:48:35,279 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:35,279 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,279 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"]
2016-01-14 15:48:35,280 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:35,280 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 2C49410EB8674D45AF855319EC703DD2, for 1 inputs
2016-01-14 15:48:35,280 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DDD4DA62B63D4F7EB1BA28AAD2B36F10, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:35,281 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@563e5f74; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@18ece4e8
2016-01-14 15:48:35,285 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:35,286 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:35,286 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,287 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786513294_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:35,287 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,288 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DDD4DA62B63D4F7EB1BA28AAD2B36F10, mem on close (mb), free: 84, total: 428, max: 672
2016-01-14 15:48:35,288 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,288 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:35,289 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:35,289 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:35,289 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:35,290 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:35 Completed running task attempt: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,291 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:35,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0, startTime=1452786515243, finishTime=1452786515291, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515281, Process_Duration=6, Process_End_Time=1452786515287, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:35,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:35,292 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:35,292 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,292 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:35,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:35,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_FINISHED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, taskId=task_1452786513294_0001_1_01_000000, startTime=1452786515243, finishTime=1452786515292, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452786513294_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515281, Process_Duration=6, Process_End_Time=1452786515287, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:35,294 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:35,294 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] : 1
2016-01-14 15:48:35,294 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:35,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786515281, Process_Duration=6, Process_End_Time=1452786515287, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:35,295 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:35,296 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:35,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_FINISHED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, vertexId=vertex_1452786513294_0001_1_01, initRequestedTime=1452786513483, initedTime=1452786513492, startRequestedTime=1452786513503, startedTime=1452786513503, finishTime=1452786515294, timeTaken=1791, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515281, Process_Duration=6, Process_End_Time=1452786515287, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786515243, firstTasksToStart=[ task_1452786513294_0001_1_01_000000 ], lastTaskFinishTime=1452786515291, lastTasksToFinish=[ task_1452786513294_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786513294_0001_1_01_000000 ], longestDurationTasks=[ task_1452786513294_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:35,297 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:35,297 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786513294_0001_00_000002
2016-01-14 15:48:35,298 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:35,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786513294_0001_00_000002, stoppedTime=1452786515298, exitStatus=0
2016-01-14 15:48:35,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:35,298 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:35,299 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:35,299 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786513294_0001_1
2016-01-14 15:48:35,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786513294_0001_1
2016-01-14 15:48:35,300 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:35,300 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8199A0D1EEF1492B8D41644EB11897FF for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:35,308 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786513294_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:35,308 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786513294_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:35,309 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786513294_0001_00_000002, Exiting
2016-01-14 15:48:35,309 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786513294_0001_00_000002
2016-01-14 15:48:35,309 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786513294_0001_00_000002
2016-01-14 15:48:35,310 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786513294_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:35,310 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786513294_0001_00_000002 completed successfully
2016-01-14 15:48:35,312 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786513294_0001_1, queueSize=0
2016-01-14 15:48:35,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:35,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786513294_0001_1, startTime=1452786513479, finishTime=1452786515309, timeTaken=1830, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=1895, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=897581056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573028869, Process_Duration=10, Process_End_Time=2905573028879, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:35,313 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786513294_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:35,313 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786513294_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:35,314 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786513294_0001_00_000002
2016-01-14 15:48:35,314 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:35 Completed Dag: dag_1452786513294_0001_1
2016-01-14 15:48:35,315 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786513294_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:35,315 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786513294_0001_1_post
2016-01-14 15:48:35,315 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,316 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:35,321 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1452786513294_0001
2016-01-14 15:48:35,323 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:35,324 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:35,324 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:35,324 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:35,324 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:35,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:35,325 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:35,325 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:35,325 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:35,326 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:35,326 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:35,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:35,327 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:35,332 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001
2016-01-14 15:48:35,333 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:02.055
2016-01-14 15:48:35,394 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:35,395 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:35,438 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 15:48:35,447 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.034
2016-01-14 15:48:35,448 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:35,448 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:35,448 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:35,460 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:35,469 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false
2016-01-14 15:48:35,508 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:35,508 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:35,508 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"]
2016-01-14 15:48:35,509 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:35,509 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:35,509 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:35,511 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_true_false
2016-01-14 15:48:35,511 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:35,519 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:35,524 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:35,524 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:35,524 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:35,533 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001 doesn't exist and is created
2016-01-14 15:48:35,548 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:35,548 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:35,549 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001
2016-01-14 15:48:35,552 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786515524_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:35,560 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:35,561 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:35,562 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:35,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:35,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001/localmode-log-dir
2016-01-14 15:48:35,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001/localmode-log-dir/history.txt.appattempt_1452786515524_0001_000000, maxErrors=10
2016-01-14 15:48:35,563 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:35,563 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786515524_0001_000000, appSubmitTime=1452786515552, launchTime=1452786515552
2016-01-14 15:48:35,566 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:35,567 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:35,569 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:35,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:35,571 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55639
2016-01-14 15:48:35,575 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:35,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786515524_0001_000000, startTime=1452786515577
2016-01-14 15:48:35,577 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:35,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:35,649 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:35,649 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:35,649 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1452786515524_0001, dagName=[AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-01-14 15:48:35,661 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-01-14 15:48:35,663 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786515524_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001/localmode-log-dir/dag_1452786515524_0001_1.dot
2016-01-14 15:48:35,663 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001/localmode-log-dir/dag_1452786515524_0001_1-tez-dag.pb.txt
2016-01-14 15:48:35,687 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786515524_0001_1
2016-01-14 15:48:35,687 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,687 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-01-14 15:48:35 Running Dag: dag_1452786515524_0001_1
2016-01-14 15:48:35,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786515524_0001_1, submitTime=1452786515661
2016-01-14 15:48:35,700 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:35,701 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,701 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:35,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:35,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786515524_0001_1, initTime=1452786515700
2016-01-14 15:48:35,702 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786515524_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:35,703 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_STARTED]: dagID=dag_1452786515524_0001_1, startTime=1452786515702
2016-01-14 15:48:35,703 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:35,703 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:35,703 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786515524_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:35,703 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8A68EE5B98D4C73B4655A05B3EDFAC4 : {A60FAE0D99484F9EA9A870B3E7488EAA={InputName=A60FAE0D99484F9EA9A870B3E7488EAA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:35,703 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A60FAE0D99484F9EA9A870B3E7488EAA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:35,703 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1452786515524_0001, dagName=[AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-01-14 15:48:35,704 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,704 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,704 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,704 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]. Starting root input initializers: 1
2016-01-14 15:48:35,705 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:35,705 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,708 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:35,708 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,709 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,709 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,709 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EBE87D8F458D4319A7C2A0AFCDA08D1F, vertexId=vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:35,709 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EBE87D8F458D4319A7C2A0AFCDA08D1F, vertexId=vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,711 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EBE87D8F458D4319A7C2A0AFCDA08D1F, vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:35,711 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EBE87D8F458D4319A7C2A0AFCDA08D1F, vertexId=vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,712 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A60FAE0D99484F9EA9A870B3E7488EAA on vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:35,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, vertexId=vertex_1452786515524_0001_1_01, initRequestedTime=1452786515705, initedTime=1452786515712, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:35,713 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:35,715 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:35,716 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A60FAE0D99484F9EA9A870B3E7488EAA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:35,716 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:35,717 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786515524_0001, with dag id: dag_1452786515524_0001_1
2016-01-14 15:48:35,717 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:35,722 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:35,722 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A60FAE0D99484F9EA9A870B3E7488EAA on vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,722 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:35,723 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A60FAE0D99484F9EA9A870B3E7488EAA=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:35,723 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] parallelism set to 1
2016-01-14 15:48:35,723 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:35,723 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786515524_0001_1_00, eventCount=1
2016-01-14 15:48:35,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, vertexId=vertex_1452786515524_0001_1_00, initRequestedTime=1452786515703, initedTime=1452786515724, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:35,724 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:35,724 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] with distanceFromRoot: 0
2016-01-14 15:48:35,725 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:35,725 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,725 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786515524_0001_1_00, startRequestedTime=1452786515725, startedTime=1452786515725
2016-01-14 15:48:35,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:35,726 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:35,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:35,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_STARTED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, taskId=task_1452786515524_0001_1_00_000000, scheduledTime=1452786515726, launchTime=1452786515726
2016-01-14 15:48:35,727 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:35,727 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786515524_0001_1_00 for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] numStartedSources: 1 numSources: 1
2016-01-14 15:48:35,728 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] with distanceFromRoot: 1
2016-01-14 15:48:35,728 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,728 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C8A68EE5B98D4C73B4655A05B3EDFAC4 in vertex: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,728 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AC8EAB16353346EF94CD2B3A8A13BF1F with 1 source tasks and 1 pending tasks
2016-01-14 15:48:35,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786515524_0001_1_01, startRequestedTime=1452786515728, startedTime=1452786515728
2016-01-14 15:48:35,729 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:35,729 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:35,730 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: C8A68EE5B98D4C73B4655A05B3EDFAC4, VertexParallelism: 1, TaskAttemptID:attempt_1452786515524_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A60FAE0D99484F9EA9A870B3E7488EAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:35,730 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:35,730 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:35,731 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:35,731 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:35,731 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:35,732 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786515524_0001_00_000001
2016-01-14 15:48:35,732 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:35,733 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786515524_0001_00_000001
2016-01-14 15:48:35,733 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:35,734 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:35,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:35,734 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786515524_0001_00_000001, launchTime=1452786515734
2016-01-14 15:48:35,735 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786515524_0001_00_000001
2016-01-14 15:48:35,737 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786515524_0001_1_00_000000_0] to container: [container_1452786515524_0001_00_000001]
2016-01-14 15:48:35,737 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:35,737 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786515524_0001_1_00_000000_0] started. Is using containerId: [container_1452786515524_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:35,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0, startTime=1452786515737, containerId=container_1452786515524_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786515524_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:35,738 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:35,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:35,738 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786515524_0001_00_000001 given task: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,738 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:35,738 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786515524_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35 Starting to run new task attempt: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,739 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,739 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,739 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:35,739 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:35,740 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:35,740 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:35,740 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:35,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: C8A68EE5B98D4C73B4655A05B3EDFAC4, VertexParallelism: 1, TaskAttemptID:attempt_1452786515524_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A60FAE0D99484F9EA9A870B3E7488EAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:35,741 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:35,741 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:35,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A60FAE0D99484F9EA9A870B3E7488EAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:35,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:35,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,742 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:35,745 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786515524_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:35,745 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:35,745 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:35,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,747 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:35,748 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:35,748 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,756 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8A68EE5B98D4C73B4655A05B3EDFAC4, ordinal: 0
2016-01-14 15:48:35,756 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8A68EE5B98D4C73B4655A05B3EDFAC4, mem on start (mb), free: 217, total: 430, max: 672
2016-01-14 15:48:35,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:35,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:35,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:35,757 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:35,758 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:35,758 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:35,758 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:35,758 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:35,759 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:35,759 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A60FAE0D99484F9EA9A870B3E7488EAA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:35,759 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AC8EAB16353346EF94CD2B3A8A13BF1F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:35,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:35,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A60FAE0D99484F9EA9A870B3E7488EAA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:35,760 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:35,760 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:35,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,760 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:35,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:35,761 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,761 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:35,762 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2fc0f146
2016-01-14 15:48:35,763 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2fc0f146
2016-01-14 15:48:35,763 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:35,763 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:35,764 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:35,764 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,764 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: A6E9E074BD1D414ABDACF2055C924228
2016-01-14 15:48:35,765 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A6E9E074BD1D414ABDACF2055C924228
2016-01-14 15:48:35,765 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@15ad5b59; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@28483b2d; comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@1e251c04; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:35,766 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:35,766 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:35,766 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:35,780 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:35,780 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:35,780 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:35,780 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:35,781 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:35,781 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:35,781 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:35,782 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8A68EE5B98D4C73B4655A05B3EDFAC4, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:35,788 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,788 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8A68EE5B98D4C73B4655A05B3EDFAC4, mem on close (mb), free: 115, total: 430, max: 672
2016-01-14 15:48:35,789 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:35,789 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:35,789 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:35,789 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:35,792 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:35 Completed running task attempt: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,797 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:35,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786515782, Process_Duration=6, Process_End_Time=1452786515788, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:35,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786515524_0001_1_00, eventCount=1
2016-01-14 15:48:35,799 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:35,799 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:35,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0, startTime=1452786515737, finishTime=1452786515799, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515782, Process_Duration=6, Process_End_Time=1452786515788, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:35,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:35,800 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786515524_0001_00_000001
2016-01-14 15:48:35,800 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,800 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:35,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:35,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_FINISHED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, taskId=task_1452786515524_0001_1_00_000000, startTime=1452786515737, finishTime=1452786515800, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1452786515524_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515782, Process_Duration=6, Process_End_Time=1452786515788, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:35,801 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:35,801 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] : 1
2016-01-14 15:48:35,801 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:35,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_FINISHED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, vertexId=vertex_1452786515524_0001_1_00, initRequestedTime=1452786515703, initedTime=1452786515724, startRequestedTime=1452786515725, startedTime=1452786515725, finishTime=1452786515801, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515782, Process_Duration=6, Process_End_Time=1452786515788, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786515737, firstTasksToStart=[ task_1452786515524_0001_1_00_000000 ], lastTaskFinishTime=1452786515799, lastTasksToFinish=[ task_1452786515524_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786515524_0001_1_00_000000 ], longestDurationTasks=[ task_1452786515524_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:35,805 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:35,805 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:35,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786515524_0001_00_000001, stoppedTime=1452786515805, exitStatus=0
2016-01-14 15:48:35,805 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:35,805 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] attempt: attempt_1452786515524_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:35,806 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,806 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,806 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:35,806 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786515524_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:35,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:35,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:35,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_STARTED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, taskId=task_1452786515524_0001_1_01_000000, scheduledTime=1452786515806, launchTime=1452786515806
2016-01-14 15:48:35,808 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:35,808 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,809 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:35,809 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: AC8EAB16353346EF94CD2B3A8A13BF1F, VertexParallelism: 1, TaskAttemptID:attempt_1452786515524_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EBE87D8F458D4319A7C2A0AFCDA08D1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:35,810 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:35,810 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:35,811 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786515524_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:35,811 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786515524_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:35,811 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786515524_0001_00_000001, Exiting
2016-01-14 15:48:35,811 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786515524_0001_00_000001
2016-01-14 15:48:35,811 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786515524_0001_00_000001
2016-01-14 15:48:35,812 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786515524_0001_00_000002
2016-01-14 15:48:35,812 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:35,813 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786515524_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:35,814 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786515524_0001_00_000001 completed successfully
2016-01-14 15:48:35,814 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:35,814 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786515524_0001_00_000002
2016-01-14 15:48:35,815 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:35,818 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:35,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:35,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786515524_0001_00_000002, launchTime=1452786515818
2016-01-14 15:48:35,819 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786515524_0001_00_000002
2016-01-14 15:48:35,820 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786515524_0001_1_01_000000_0] to container: [container_1452786515524_0001_00_000002]
2016-01-14 15:48:35,820 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:35,821 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786515524_0001_1_01_000000_0] started. Is using containerId: [container_1452786515524_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:35,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0, startTime=1452786515820, containerId=container_1452786515524_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786515524_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:35,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:35,821 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:35,821 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:35,821 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786515524_0001_00_000002 given task: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,822 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786515524_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35 Starting to run new task attempt: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,822 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,822 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,822 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:35,822 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:35,823 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:35,823 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:35,823 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:35,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: AC8EAB16353346EF94CD2B3A8A13BF1F, VertexParallelism: 1, TaskAttemptID:attempt_1452786515524_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EBE87D8F458D4319A7C2A0AFCDA08D1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:35,823 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:35,824 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:35,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:35,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,828 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786515524_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:35,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EBE87D8F458D4319A7C2A0AFCDA08D1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:35,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:35,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EBE87D8F458D4319A7C2A0AFCDA08D1F
2016-01-14 15:48:35,830 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:35,830 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EBE87D8F458D4319A7C2A0AFCDA08D1F
2016-01-14 15:48:35,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:35,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,832 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:35,834 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:35,834 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:35,835 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,840 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AC8EAB16353346EF94CD2B3A8A13BF1F, ordinal: 1
2016-01-14 15:48:35,840 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC8EAB16353346EF94CD2B3A8A13BF1F, mem on start (mb), free: 107, total: 430, max: 672
2016-01-14 15:48:35,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:35,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:35,843 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:35,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EBE87D8F458D4319A7C2A0AFCDA08D1F
2016-01-14 15:48:35,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:35,844 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:35,844 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:35,844 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:35,844 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:35,844 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:35,845 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:35,845 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EBE87D8F458D4319A7C2A0AFCDA08D1F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:35,845 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C8A68EE5B98D4C73B4655A05B3EDFAC4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:35,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:35,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C8A68EE5B98D4C73B4655A05B3EDFAC4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:35,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,845 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:35,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:35,848 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:35,848 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:35,849 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C8A68EE5B98D4C73B4655A05B3EDFAC4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:35,849 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:35,849 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:35,850 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:35,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:35,851 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,851 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786515524_0001_1_00_000000_0_10035, runDuration: 48607, ]
2016-01-14 15:48:35,853 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786515524_0001_1_00_000000_0_10035] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786515853, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:35,853 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,854 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:35,855 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C8A68EE5B98D4C73B4655A05B3EDFAC4] #7 in 3ms
2016-01-14 15:48:35,856 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:35,857 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: A6E9E074BD1D414ABDACF2055C924228
2016-01-14 15:48:35,857 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"], id: EBE87D8F458D4319A7C2A0AFCDA08D1F
2016-01-14 15:48:35,857 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"]
2016-01-14 15:48:35,857 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A6E9E074BD1D414ABDACF2055C924228, for 1 inputs
2016-01-14 15:48:35,857 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C8A68EE5B98D4C73B4655A05B3EDFAC4] #7, status:false, isInterrupted:false
2016-01-14 15:48:35,859 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:35,859 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:35,859 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:35,860 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:35,860 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:35,860 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,861 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AC8EAB16353346EF94CD2B3A8A13BF1F, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:48:35,861 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:35,861 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3ae0242f; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5167a7db
2016-01-14 15:48:35,865 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:35,866 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:35,866 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,866 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786515524_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:35,867 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,867 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC8EAB16353346EF94CD2B3A8A13BF1F, mem on close (mb), free: 103, total: 430, max: 672
2016-01-14 15:48:35,867 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,867 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:35,868 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:35,868 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:35,868 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:35,868 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:35 Completed running task attempt: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,869 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:35,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:35,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0, startTime=1452786515820, finishTime=1452786515869, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515861, Process_Duration=6, Process_End_Time=1452786515867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:35,870 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:35,870 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,870 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:35,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:35,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_FINISHED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, taskId=task_1452786515524_0001_1_01_000000, startTime=1452786515820, finishTime=1452786515870, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1452786515524_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515861, Process_Duration=6, Process_End_Time=1452786515867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:35,871 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:35,871 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] : 1
2016-01-14 15:48:35,871 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:35,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_FINISHED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, vertexId=vertex_1452786515524_0001_1_01, initRequestedTime=1452786515705, initedTime=1452786515712, startRequestedTime=1452786515728, startedTime=1452786515728, finishTime=1452786515871, timeTaken=143, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515861, Process_Duration=6, Process_End_Time=1452786515867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786515820, firstTasksToStart=[ task_1452786515524_0001_1_01_000000 ], lastTaskFinishTime=1452786515869, lastTasksToFinish=[ task_1452786515524_0001_1_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786515524_0001_1_01_000000 ], longestDurationTasks=[ task_1452786515524_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:35,873 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:35,873 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:35,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786515524_0001_00_000002, stoppedTime=1452786515873, exitStatus=0
2016-01-14 15:48:35,873 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:35,874 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:35,874 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:35,874 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786515524_0001_1
2016-01-14 15:48:35,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786515524_0001_1
2016-01-14 15:48:35,874 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,874 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EBE87D8F458D4319A7C2A0AFCDA08D1F for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786515861, Process_Duration=6, Process_End_Time=1452786515867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:35,875 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:35,875 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,876 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786515524_0001_00_000002
2016-01-14 15:48:35,876 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786515524_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:35,876 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786515524_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:35,877 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786515524_0001_00_000002, Exiting
2016-01-14 15:48:35,877 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786515524_0001_00_000002
2016-01-14 15:48:35,877 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786515524_0001_00_000002
2016-01-14 15:48:35,878 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786515524_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:35,878 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786515524_0001_00_000002 completed successfully
2016-01-14 15:48:35,886 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786515524_0001_1, queueSize=0
2016-01-14 15:48:35,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:35,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786515524_0001_1, startTime=1452786515702, finishTime=1452786515884, timeTaken=182, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=859, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=901775360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573031643, Process_Duration=12, Process_End_Time=2905573031655, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:35,887 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786515524_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:35,887 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786515524_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:35,887 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:35,887 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786515524_0001_00_000002
2016-01-14 15:48:35 Completed Dag: dag_1452786515524_0001_1
2016-01-14 15:48:35,887 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786515524_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:35,887 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786515524_0001_1_post
2016-01-14 15:48:35,887 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,888 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:35,891 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1452786515524_0001
2016-01-14 15:48:35,894 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:35,895 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:35,895 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:35,895 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:35,895 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:35,895 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:35,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:35,896 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:35,896 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:35,896 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:35,897 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:35,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:35,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:35,902 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001
2016-01-14 15:48:35,903 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.394
2016-01-14 15:48:35,957 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:35,957 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:35,993 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 15:48:36,005 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-01-14 15:48:36,006 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:36,006 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:36,006 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:36,026 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:36,037 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false
2016-01-14 15:48:36,076 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:36,076 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:36,076 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"]
2016-01-14 15:48:36,076 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:36,077 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:36,077 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:36,078 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_true_false
2016-01-14 15:48:36,078 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:36,087 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:36,091 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:36,091 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:36,091 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:36,100 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001 doesn't exist and is created
2016-01-14 15:48:36,120 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:36,121 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:36,121 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001
2016-01-14 15:48:36,124 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786516091_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:36,137 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:36,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:36,140 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:36,140 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:36,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001/localmode-log-dir
2016-01-14 15:48:36,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001/localmode-log-dir/history.txt.appattempt_1452786516091_0001_000000, maxErrors=10
2016-01-14 15:48:36,141 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:36,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786516091_0001_000000, appSubmitTime=1452786516124, launchTime=1452786516124
2016-01-14 15:48:36,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:36,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:36,148 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:36,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:36,153 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:48679
2016-01-14 15:48:36,155 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:36,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786516091_0001_000000, startTime=1452786516157
2016-01-14 15:48:36,157 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:36,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:36,221 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:36,221 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:36,221 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1452786516091_0001, dagName=[73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-01-14 15:48:36,235 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-01-14 15:48:36,236 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786516091_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001/localmode-log-dir/dag_1452786516091_0001_1.dot
2016-01-14 15:48:36,237 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001/localmode-log-dir/dag_1452786516091_0001_1-tez-dag.pb.txt
2016-01-14 15:48:36,259 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786516091_0001_1
2016-01-14 15:48:36,260 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:36,260 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-01-14 15:48:36 Running Dag: dag_1452786516091_0001_1
2016-01-14 15:48:36,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:36,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786516091_0001_1, submitTime=1452786516235
2016-01-14 15:48:36,279 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,279 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:36,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:36,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786516091_0001_1, initTime=1452786516278
2016-01-14 15:48:36,281 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786516091_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:36,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_STARTED]: dagID=dag_1452786516091_0001_1, startTime=1452786516282
2016-01-14 15:48:36,282 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:36,282 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786516091_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:36,282 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:36,283 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BCA7656A60EE4599B0476B1E8894585D : {EF76503B14824744B92323B77FD4D5A5={InputName=EF76503B14824744B92323B77FD4D5A5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:36,283 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EF76503B14824744B92323B77FD4D5A5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:36,283 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,283 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,283 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,284 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]. Starting root input initializers: 1
2016-01-14 15:48:36,284 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1452786516091_0001, dagName=[73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-01-14 15:48:36,285 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:36,285 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,287 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EF76503B14824744B92323B77FD4D5A5 on vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,288 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:36,289 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,289 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:36,290 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,290 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,290 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0CE4D6ED85304C68A60E06871115D775, vertexId=vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:36,291 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0CE4D6ED85304C68A60E06871115D775, vertexId=vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,292 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EF76503B14824744B92323B77FD4D5A5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:36,293 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:36,293 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:36,294 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0CE4D6ED85304C68A60E06871115D775, vertexName=B14CEFDAAB9446D9B508118DD3D287B2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:36,294 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0CE4D6ED85304C68A60E06871115D775, vertexId=vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:36,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, vertexId=vertex_1452786516091_0001_1_01, initRequestedTime=1452786516285, initedTime=1452786516296, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:36,297 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:36,297 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:36,297 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EF76503B14824744B92323B77FD4D5A5 on vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,298 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:36,298 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EF76503B14824744B92323B77FD4D5A5=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:36,299 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] parallelism set to 1
2016-01-14 15:48:36,299 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:36,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786516091_0001_1_00, eventCount=1
2016-01-14 15:48:36,299 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786516091_0001, with dag id: dag_1452786516091_0001_1
2016-01-14 15:48:36,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:36,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, vertexId=vertex_1452786516091_0001_1_00, initRequestedTime=1452786516283, initedTime=1452786516302, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:36,304 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] with distanceFromRoot: 0
2016-01-14 15:48:36,304 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:36,304 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:36,305 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786516091_0001_1_00, startRequestedTime=1452786516304, startedTime=1452786516304
2016-01-14 15:48:36,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:36,305 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:36,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_STARTED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, taskId=task_1452786516091_0001_1_00_000000, scheduledTime=1452786516306, launchTime=1452786516306
2016-01-14 15:48:36,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:36,306 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:36,309 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786516091_0001_1_00 for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] numStartedSources: 1 numSources: 1
2016-01-14 15:48:36,309 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] with distanceFromRoot: 1
2016-01-14 15:48:36,310 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,310 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BCA7656A60EE4599B0476B1E8894585D in vertex: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,311 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B14CEFDAAB9446D9B508118DD3D287B2 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:36,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:36,311 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786516091_0001_1_01, startRequestedTime=1452786516310, startedTime=1452786516310
2016-01-14 15:48:36,311 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:36,312 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: BCA7656A60EE4599B0476B1E8894585D, VertexParallelism: 1, TaskAttemptID:attempt_1452786516091_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF76503B14824744B92323B77FD4D5A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B14CEFDAAB9446D9B508118DD3D287B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:36,313 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:36,313 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:36,313 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:36,314 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:36,314 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:36,315 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786516091_0001_00_000001
2016-01-14 15:48:36,315 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:36,317 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786516091_0001_00_000001
2016-01-14 15:48:36,317 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:36,317 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:36,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786516091_0001_00_000001, launchTime=1452786516317
2016-01-14 15:48:36,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:36,319 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786516091_0001_00_000001
2016-01-14 15:48:36,320 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786516091_0001_1_00_000000_0] to container: [container_1452786516091_0001_00_000001]
2016-01-14 15:48:36,320 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:36,321 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786516091_0001_00_000001 given task: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,321 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786516091_0001_1_00_000000_0] started. Is using containerId: [container_1452786516091_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:36,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0, startTime=1452786516321, containerId=container_1452786516091_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786516091_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:36,321 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:36,321 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:36,321 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786516091_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36 Starting to run new task attempt: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:36,322 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,322 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:36,322 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:36,323 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:36,323 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:36,323 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:36,323 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:36,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: BCA7656A60EE4599B0476B1E8894585D, VertexParallelism: 1, TaskAttemptID:attempt_1452786516091_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF76503B14824744B92323B77FD4D5A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B14CEFDAAB9446D9B508118DD3D287B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:36,323 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:36,324 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:36,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:36,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EF76503B14824744B92323B77FD4D5A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:36,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:36,329 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:36,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B14CEFDAAB9446D9B508118DD3D287B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:36,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:36,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,332 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:36,332 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,333 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:36,335 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,335 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,326 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786516091_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:36,341 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:36,341 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,344 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BCA7656A60EE4599B0476B1E8894585D, ordinal: 0
2016-01-14 15:48:36,344 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCA7656A60EE4599B0476B1E8894585D, mem on start (mb), free: 232, total: 420, max: 672
2016-01-14 15:48:36,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:36,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:36,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:36,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:36,345 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:36,345 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:36,345 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:36,346 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:36,346 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:36,347 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:36,347 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B14CEFDAAB9446D9B508118DD3D287B2, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:36,347 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EF76503B14824744B92323B77FD4D5A5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:36,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:36,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,348 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:36,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EF76503B14824744B92323B77FD4D5A5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:36,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:36,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:36,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:36,351 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,351 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:36,352 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1216bde8
2016-01-14 15:48:36,353 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1216bde8
2016-01-14 15:48:36,353 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:36,353 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:36,356 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:36,356 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,356 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: DA36AE3B34AF49A88E76AF2FE760DC0A
2016-01-14 15:48:36,357 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] DA36AE3B34AF49A88E76AF2FE760DC0A
2016-01-14 15:48:36,358 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5d571114; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7f2267eb; comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@2bd4b682; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:36,359 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:36,359 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:36,360 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:36,373 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:36,374 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:36,374 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:36,374 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:36,374 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:36,375 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:36,375 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:36,375 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BCA7656A60EE4599B0476B1E8894585D, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:36,382 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,382 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCA7656A60EE4599B0476B1E8894585D, mem on close (mb), free: 130, total: 420, max: 672
2016-01-14 15:48:36,382 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:36,382 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:36,382 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:36,382 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:37,444 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:37,445 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:38,028 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:38 Completed running task attempt: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:38,034 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:38,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786516376, Process_Duration=6, Process_End_Time=1452786516382, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:38,035 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:38,036 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786516091_0001_1_00, eventCount=1
2016-01-14 15:48:38,036 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786516091_0001_00_000001
2016-01-14 15:48:38,037 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:38,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0, startTime=1452786516321, finishTime=1452786518036, timeTaken=1715, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786516376, Process_Duration=6, Process_End_Time=1452786516382, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:38,038 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:38,038 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:38,038 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:38,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:38,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_FINISHED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, taskId=task_1452786516091_0001_1_00_000000, startTime=1452786516321, finishTime=1452786518038, timeTaken=1717, status=SUCCEEDED, successfulAttemptID=attempt_1452786516091_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786516376, Process_Duration=6, Process_End_Time=1452786516382, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:38,039 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:38,039 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] : 1
2016-01-14 15:48:38,039 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:38,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_FINISHED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, vertexId=vertex_1452786516091_0001_1_00, initRequestedTime=1452786516283, initedTime=1452786516302, startRequestedTime=1452786516304, startedTime=1452786516304, finishTime=1452786518039, timeTaken=1735, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786516376, Process_Duration=6, Process_End_Time=1452786516382, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786516321, firstTasksToStart=[ task_1452786516091_0001_1_00_000000 ], lastTaskFinishTime=1452786518036, lastTasksToFinish=[ task_1452786516091_0001_1_00_000000 ], minTaskDuration=1715, maxTaskDuration=1715, avgTaskDuration=1715.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786516091_0001_1_00_000000 ], longestDurationTasks=[ task_1452786516091_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:38,041 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:38,041 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:38,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786516091_0001_00_000001, stoppedTime=1452786518041, exitStatus=0
2016-01-14 15:48:38,041 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:38,041 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] attempt: attempt_1452786516091_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:38,041 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:38,042 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:38,042 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:38,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:38,043 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:38,044 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786516091_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:38,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_STARTED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, taskId=task_1452786516091_0001_1_01_000000, scheduledTime=1452786518044, launchTime=1452786518044
2016-01-14 15:48:38,044 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:38,045 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:38,045 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:38,046 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: B14CEFDAAB9446D9B508118DD3D287B2, VertexParallelism: 1, TaskAttemptID:attempt_1452786516091_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BCA7656A60EE4599B0476B1E8894585D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0CE4D6ED85304C68A60E06871115D775, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:38,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:38,046 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:38,047 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:38,047 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786516091_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:38,047 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786516091_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:38,047 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786516091_0001_00_000001, Exiting
2016-01-14 15:48:38,047 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786516091_0001_00_000001
2016-01-14 15:48:38,048 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786516091_0001_00_000001
2016-01-14 15:48:38,048 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786516091_0001_00_000002
2016-01-14 15:48:38,050 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:38,051 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786516091_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:38,051 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786516091_0001_00_000002
2016-01-14 15:48:38,051 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786516091_0001_00_000001 completed successfully
2016-01-14 15:48:38,052 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:38,052 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:38,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:38,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786516091_0001_00_000002, launchTime=1452786518054
2016-01-14 15:48:38,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:38,055 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786516091_0001_00_000002
2016-01-14 15:48:38,056 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786516091_0001_1_01_000000_0] to container: [container_1452786516091_0001_00_000002]
2016-01-14 15:48:38,056 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:38,056 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786516091_0001_00_000002 given task: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,056 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786516091_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,056 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786516091_0001_1_01_000000_0] started. Is using containerId: [container_1452786516091_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:38 Starting to run new task attempt: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,056 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:38,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0, startTime=1452786518056, containerId=container_1452786516091_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786516091_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:38,057 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:38,057 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:38,058 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,059 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:38,059 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:38,059 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:38,059 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:38,059 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:38,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: B14CEFDAAB9446D9B508118DD3D287B2, VertexParallelism: 1, TaskAttemptID:attempt_1452786516091_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BCA7656A60EE4599B0476B1E8894585D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0CE4D6ED85304C68A60E06871115D775, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:38,060 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:38,061 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:38,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,062 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786516091_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:38,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BCA7656A60EE4599B0476B1E8894585D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:38,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0CE4D6ED85304C68A60E06871115D775, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:38,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:38,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:38,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0CE4D6ED85304C68A60E06871115D775
2016-01-14 15:48:38,064 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:38,065 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0CE4D6ED85304C68A60E06871115D775
2016-01-14 15:48:38,066 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:38,068 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:38,068 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:38,068 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,077 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B14CEFDAAB9446D9B508118DD3D287B2, ordinal: 1
2016-01-14 15:48:38,077 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B14CEFDAAB9446D9B508118DD3D287B2, mem on start (mb), free: 254, total: 432, max: 672
2016-01-14 15:48:38,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:38,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:38,079 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:38,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0CE4D6ED85304C68A60E06871115D775
2016-01-14 15:48:38,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:38,079 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:38,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:38,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:38,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:38,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:38,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:38,080 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0CE4D6ED85304C68A60E06871115D775, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:38,080 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BCA7656A60EE4599B0476B1E8894585D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:38,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:38,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BCA7656A60EE4599B0476B1E8894585D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:38,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:38,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:38,082 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:38,082 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:38,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BCA7656A60EE4599B0476B1E8894585D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:38,083 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:38,083 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:38,083 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:38,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:38,084 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,086 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786516091_0001_1_00_000000_0_10040, runDuration: 1701074, ]
2016-01-14 15:48:38,087 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:38,088 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: DA36AE3B34AF49A88E76AF2FE760DC0A
2016-01-14 15:48:38,088 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786516091_0001_1_00_000000_0_10040] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786518088, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:38,088 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"], id: 0CE4D6ED85304C68A60E06871115D775
2016-01-14 15:48:38,088 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,088 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"]
2016-01-14 15:48:38,089 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] DA36AE3B34AF49A88E76AF2FE760DC0A, for 1 inputs
2016-01-14 15:48:38,088 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:38,090 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BCA7656A60EE4599B0476B1E8894585D] #8 in 3ms
2016-01-14 15:48:38,090 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BCA7656A60EE4599B0476B1E8894585D] #8, status:false, isInterrupted:false
2016-01-14 15:48:38,091 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:38,092 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:38,092 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:38,092 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:38,092 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:38,092 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,092 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:38,094 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B14CEFDAAB9446D9B508118DD3D287B2, all 1 inputs ready in: 00:00:00.004
2016-01-14 15:48:38,096 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@36db6ea8; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3df3bb
2016-01-14 15:48:38,100 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:38,100 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:38,101 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,101 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786516091_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:38,102 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,102 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B14CEFDAAB9446D9B508118DD3D287B2, mem on close (mb), free: 249, total: 432, max: 672
2016-01-14 15:48:38,102 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:38,102 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:38,102 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,103 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:38,103 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:38,103 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:38,104 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:38,104 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:38 Completed running task attempt: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,105 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:38,107 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786518094, Process_Duration=8, Process_End_Time=1452786518102, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:38,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0, startTime=1452786518056, finishTime=1452786518107, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518094, Process_Duration=8, Process_End_Time=1452786518102, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:38,107 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:38,107 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,107 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:38,107 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:38,108 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:38,108 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:38,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_FINISHED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, taskId=task_1452786516091_0001_1_01_000000, startTime=1452786518056, finishTime=1452786518108, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1452786516091_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518094, Process_Duration=8, Process_End_Time=1452786518102, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:38,109 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:38,109 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] : 1
2016-01-14 15:48:38,109 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:38,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:38,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_FINISHED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, vertexId=vertex_1452786516091_0001_1_01, initRequestedTime=1452786516285, initedTime=1452786516296, startRequestedTime=1452786516310, startedTime=1452786516310, finishTime=1452786518109, timeTaken=1799, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518094, Process_Duration=8, Process_End_Time=1452786518102, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786518056, firstTasksToStart=[ task_1452786516091_0001_1_01_000000 ], lastTaskFinishTime=1452786518107, lastTasksToFinish=[ task_1452786516091_0001_1_01_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786516091_0001_1_01_000000 ], longestDurationTasks=[ task_1452786516091_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:38,110 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:38,111 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:38,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786516091_0001_00_000002, stoppedTime=1452786518111, exitStatus=0
2016-01-14 15:48:38,111 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:38,108 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786516091_0001_00_000002
2016-01-14 15:48:38,111 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:38,112 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:38,112 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786516091_0001_1
2016-01-14 15:48:38,112 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786516091_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:38,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786516091_0001_1
2016-01-14 15:48:38,112 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0CE4D6ED85304C68A60E06871115D775 for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:38,116 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786516091_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:38,116 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786516091_0001_00_000002, Exiting
2016-01-14 15:48:38,116 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786516091_0001_00_000002
2016-01-14 15:48:38,116 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786516091_0001_00_000002
2016-01-14 15:48:38,117 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786516091_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:38,118 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786516091_0001_00_000002 completed successfully
2016-01-14 15:48:38,122 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:38,124 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786516091_0001_1, queueSize=0
2016-01-14 15:48:38,124 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:38,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786516091_0001_1, startTime=1452786516282, finishTime=1452786518122, timeTaken=1840, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=859, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=905969664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573034470, Process_Duration=14, Process_End_Time=2905573034484, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:38,126 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786516091_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:38,126 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786516091_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:38,127 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786516091_0001_00_000002
2016-01-14 15:48:38,127 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:38 Completed Dag: dag_1452786516091_0001_1
2016-01-14 15:48:38,130 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786516091_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:38,130 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786516091_0001_1_post
2016-01-14 15:48:38,130 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,130 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:38,131 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1452786516091_0001
2016-01-14 15:48:38,133 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:38,133 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:38,133 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:38,133 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:38,133 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:38,134 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:38,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:38,134 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:38,135 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:38,135 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:38,135 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:38,136 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:38,136 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:38,140 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001
2016-01-14 15:48:38,141 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:02.065
2016-01-14 15:48:38,194 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:38,194 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:38,240 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 15:48:38,253 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.040
2016-01-14 15:48:38,254 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:38,254 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:38,254 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:38,266 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:38,274 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false
2016-01-14 15:48:38,310 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:38,311 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:38,311 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"]
2016-01-14 15:48:38,312 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:38,312 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:38,313 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:38,314 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...e/false_false_false_false
2016-01-14 15:48:38,314 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:38,322 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:38,327 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:38,327 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:38,327 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:38,338 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001 doesn't exist and is created
2016-01-14 15:48:38,356 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:38,356 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:38,357 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001
2016-01-14 15:48:38,361 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786518327_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:38,369 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:38,370 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:38,372 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:38,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:38,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001/localmode-log-dir
2016-01-14 15:48:38,374 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001/localmode-log-dir/history.txt.appattempt_1452786518327_0001_000000, maxErrors=10
2016-01-14 15:48:38,374 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:38,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786518327_0001_000000, appSubmitTime=1452786518361, launchTime=1452786518361
2016-01-14 15:48:38,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:38,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:38,383 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:38,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:38,388 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40196
2016-01-14 15:48:38,390 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:38,392 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786518327_0001_000000, startTime=1452786518392
2016-01-14 15:48:38,392 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:38,392 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:38,457 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:38,457 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:38,457 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1452786518327_0001, dagName=[E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false
2016-01-14 15:48:38,470 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false
2016-01-14 15:48:38,471 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786518327_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001/localmode-log-dir/dag_1452786518327_0001_1.dot
2016-01-14 15:48:38,472 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001/localmode-log-dir/dag_1452786518327_0001_1-tez-dag.pb.txt
2016-01-14 15:48:38,490 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786518327_0001_1
2016-01-14 15:48:38,490 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,490 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false
2016-01-14 15:48:38 Running Dag: dag_1452786518327_0001_1
2016-01-14 15:48:38,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:38,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786518327_0001_1, submitTime=1452786518470
2016-01-14 15:48:38,505 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,506 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:38,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:38,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786518327_0001_1, initTime=1452786518505
2016-01-14 15:48:38,507 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518327_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:38,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_STARTED]: dagID=dag_1452786518327_0001_1, startTime=1452786518507
2016-01-14 15:48:38,508 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:38,508 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518327_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:38,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:38,508 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EF1ACD2574CE4C67966F6DB70D92535D : {3FC9FD5A60764766900DB085D1ABA2C1={InputName=3FC9FD5A60764766900DB085D1ABA2C1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:38,509 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3FC9FD5A60764766900DB085D1ABA2C1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:38,509 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1452786518327_0001, dagName=[E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false
2016-01-14 15:48:38,509 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,509 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,509 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,510 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]. Starting root input initializers: 1
2016-01-14 15:48:38,510 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:38,510 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3FC9FD5A60764766900DB085D1ABA2C1 on vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,510 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,514 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:38,514 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3FC9FD5A60764766900DB085D1ABA2C1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:38,514 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:38,515 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,515 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:38,515 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,515 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,516 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9DC90CBA103E4575BC5169E30B287DA7, vertexId=vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:38,516 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9DC90CBA103E4575BC5169E30B287DA7, vertexId=vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,515 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:38,518 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9DC90CBA103E4575BC5169E30B287DA7, vertexName=B720CBD51C0C48DF97E86274C0E700E4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:38,518 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9DC90CBA103E4575BC5169E30B287DA7, vertexId=vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,518 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:38,519 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3FC9FD5A60764766900DB085D1ABA2C1 on vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,520 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:38,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, vertexId=vertex_1452786518327_0001_1_01, initRequestedTime=1452786518510, initedTime=1452786518520, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:38,521 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:38,521 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:38,521 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3FC9FD5A60764766900DB085D1ABA2C1=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:38,521 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] parallelism set to 1
2016-01-14 15:48:38,521 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:38,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786518327_0001_1_00, eventCount=1
2016-01-14 15:48:38,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, vertexId=vertex_1452786518327_0001_1_00, initRequestedTime=1452786518508, initedTime=1452786518522, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:38,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:38,523 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] with distanceFromRoot: 0
2016-01-14 15:48:38,523 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:38,523 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,523 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:38,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786518327_0001_1_00, startRequestedTime=1452786518523, startedTime=1452786518523
2016-01-14 15:48:38,524 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:38,524 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:38,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_STARTED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, taskId=task_1452786518327_0001_1_00_000000, scheduledTime=1452786518524, launchTime=1452786518524
2016-01-14 15:48:38,525 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:38,525 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786518327_0001_1_00 for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] numStartedSources: 1 numSources: 1
2016-01-14 15:48:38,525 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] with distanceFromRoot: 1
2016-01-14 15:48:38,526 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,526 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EF1ACD2574CE4C67966F6DB70D92535D in vertex: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,526 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B720CBD51C0C48DF97E86274C0E700E4 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:38,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:38,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786518327_0001_1_01, startRequestedTime=1452786518526, startedTime=1452786518526
2016-01-14 15:48:38,527 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:38,527 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: EF1ACD2574CE4C67966F6DB70D92535D, VertexParallelism: 1, TaskAttemptID:attempt_1452786518327_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3FC9FD5A60764766900DB085D1ABA2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B720CBD51C0C48DF97E86274C0E700E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:38,527 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:38,528 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:38,528 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:38,529 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:38,529 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:38,529 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786518327_0001, with dag id: dag_1452786518327_0001_1
2016-01-14 15:48:38,529 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786518327_0001_00_000001
2016-01-14 15:48:38,530 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:38,531 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786518327_0001_00_000001
2016-01-14 15:48:38,531 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:38,532 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:38,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:38,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786518327_0001_00_000001, launchTime=1452786518532
2016-01-14 15:48:38,536 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518327_0001_00_000001
2016-01-14 15:48:38,537 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786518327_0001_1_00_000000_0] to container: [container_1452786518327_0001_00_000001]
2016-01-14 15:48:38,537 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:38,537 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786518327_0001_1_00_000000_0] started. Is using containerId: [container_1452786518327_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:38,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0, startTime=1452786518537, containerId=container_1452786518327_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786518327_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:38,538 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:38,538 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:38,538 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:38,538 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786518327_0001_00_000001 given task: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,538 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518327_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38 Starting to run new task attempt: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,539 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,539 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,539 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:38,539 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:38,539 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:38,539 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:38,540 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:38,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: EF1ACD2574CE4C67966F6DB70D92535D, VertexParallelism: 1, TaskAttemptID:attempt_1452786518327_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3FC9FD5A60764766900DB085D1ABA2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B720CBD51C0C48DF97E86274C0E700E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:38,540 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:38,540 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:38,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,543 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786518327_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:38,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3FC9FD5A60764766900DB085D1ABA2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:38,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B720CBD51C0C48DF97E86274C0E700E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:38,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:38,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:38,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,544 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,546 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:38,546 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:38,546 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,549 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:38,549 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,553 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EF1ACD2574CE4C67966F6DB70D92535D, ordinal: 0
2016-01-14 15:48:38,553 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EF1ACD2574CE4C67966F6DB70D92535D, mem on start (mb), free: 160, total: 432, max: 672
2016-01-14 15:48:38,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:38,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:38,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:38,554 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:38,554 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:38,554 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:38,554 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:38,555 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:38,555 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:38,555 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3FC9FD5A60764766900DB085D1ABA2C1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:38,555 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B720CBD51C0C48DF97E86274C0E700E4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:38,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:38,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3FC9FD5A60764766900DB085D1ABA2C1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:38,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,556 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:38,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:38,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:38,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:38,557 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,557 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:38,558 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4f7ebeec
2016-01-14 15:48:38,559 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4f7ebeec
2016-01-14 15:48:38,559 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:38,560 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:38,561 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:38,562 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,562 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 2ED7AC534669403FB9B25EDE2400DAD6
2016-01-14 15:48:38,563 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 2ED7AC534669403FB9B25EDE2400DAD6
2016-01-14 15:48:38,564 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@34adeb80; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6403b85; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@63039bf7; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:38,565 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:38,565 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:38,565 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:38,584 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:38,585 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:38,585 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:38,586 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:38,586 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:38,587 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:38,587 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:38,588 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EF1ACD2574CE4C67966F6DB70D92535D, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:38,592 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,592 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EF1ACD2574CE4C67966F6DB70D92535D, mem on close (mb), free: 202, total: 432, max: 672
2016-01-14 15:48:38,593 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:38,593 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:38,593 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:38,594 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:38,597 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:38 Completed running task attempt: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,601 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:38,602 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786518327_0001_1_00, eventCount=1
2016-01-14 15:48:38,602 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:38,602 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0, startTime=1452786518537, finishTime=1452786518602, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518588, Process_Duration=4, Process_End_Time=1452786518592, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:38,603 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:38,603 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:38,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_FINISHED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, taskId=task_1452786518327_0001_1_00_000000, startTime=1452786518537, finishTime=1452786518603, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1452786518327_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518588, Process_Duration=4, Process_End_Time=1452786518592, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:38,604 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:38,604 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] : 1
2016-01-14 15:48:38,604 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,605 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:38,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_FINISHED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, vertexId=vertex_1452786518327_0001_1_00, initRequestedTime=1452786518508, initedTime=1452786518522, startRequestedTime=1452786518523, startedTime=1452786518523, finishTime=1452786518604, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518588, Process_Duration=4, Process_End_Time=1452786518592, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786518537, firstTasksToStart=[ task_1452786518327_0001_1_00_000000 ], lastTaskFinishTime=1452786518602, lastTasksToFinish=[ task_1452786518327_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786518327_0001_1_00_000000 ], longestDurationTasks=[ task_1452786518327_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:38,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:38,606 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:38,606 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] attempt: attempt_1452786518327_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:38,606 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,606 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,607 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:38,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786518327_0001_00_000001, stoppedTime=1452786518607, exitStatus=0
2016-01-14 15:48:38,607 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:38,607 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:38,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_STARTED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, taskId=task_1452786518327_0001_1_01_000000, scheduledTime=1452786518607, launchTime=1452786518607
2016-01-14 15:48:38,608 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:38,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:38,608 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786518327_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:38,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:38,608 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,608 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:38,609 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: B720CBD51C0C48DF97E86274C0E700E4, VertexParallelism: 1, TaskAttemptID:attempt_1452786518327_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF1ACD2574CE4C67966F6DB70D92535D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9DC90CBA103E4575BC5169E30B287DA7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:38,609 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:38,609 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:38,608 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786518588, Process_Duration=4, Process_End_Time=1452786518592, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:38,609 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:38,609 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,612 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786518327_0001_00_000002
2016-01-14 15:48:38,612 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:38,614 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786518327_0001_00_000002
2016-01-14 15:48:38,614 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:38,610 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518327_0001_00_000001
2016-01-14 15:48:38,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786518327_0001_00_000002, launchTime=1452786518614
2016-01-14 15:48:38,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:38,615 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:38,616 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518327_0001_00_000002
2016-01-14 15:48:38,615 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786518327_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:38,617 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518327_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:38,617 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786518327_0001_1_01_000000_0] to container: [container_1452786518327_0001_00_000002]
2016-01-14 15:48:38,617 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786518327_0001_00_000001, Exiting
2016-01-14 15:48:38,617 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786518327_0001_00_000001
2016-01-14 15:48:38,617 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786518327_0001_00_000001
2016-01-14 15:48:38,617 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:38,619 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786518327_0001_00_000002 given task: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,619 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518327_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,619 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786518327_0001_1_01_000000_0] started. Is using containerId: [container_1452786518327_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:38 Starting to run new task attempt: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,619 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0, startTime=1452786518619, containerId=container_1452786518327_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786518327_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:38,619 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:38,620 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,620 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:38,620 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:38,620 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:38,620 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:38,621 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:38,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: B720CBD51C0C48DF97E86274C0E700E4, VertexParallelism: 1, TaskAttemptID:attempt_1452786518327_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF1ACD2574CE4C67966F6DB70D92535D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9DC90CBA103E4575BC5169E30B287DA7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:38,621 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:38,621 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786518327_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:38,622 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786518327_0001_00_000001 completed successfully
2016-01-14 15:48:38,620 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:38,622 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786518327_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:38,622 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:38,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,624 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:38,624 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:38,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EF1ACD2574CE4C67966F6DB70D92535D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:38,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9DC90CBA103E4575BC5169E30B287DA7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:38,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:38,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9DC90CBA103E4575BC5169E30B287DA7
2016-01-14 15:48:38,626 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:38,627 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9DC90CBA103E4575BC5169E30B287DA7
2016-01-14 15:48:38,627 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:38,628 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:38,628 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,630 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:38,630 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:38,631 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,636 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B720CBD51C0C48DF97E86274C0E700E4, ordinal: 1
2016-01-14 15:48:38,636 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B720CBD51C0C48DF97E86274C0E700E4, mem on start (mb), free: 195, total: 432, max: 672
2016-01-14 15:48:38,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:38,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:38,639 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:38,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9DC90CBA103E4575BC5169E30B287DA7
2016-01-14 15:48:38,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:38,640 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:38,640 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:38,640 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:38,640 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:38,640 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:38,641 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:38,641 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9DC90CBA103E4575BC5169E30B287DA7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:38,641 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EF1ACD2574CE4C67966F6DB70D92535D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:38,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:38,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EF1ACD2574CE4C67966F6DB70D92535D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:38,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:38,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:38,642 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:38,642 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:38,643 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EF1ACD2574CE4C67966F6DB70D92535D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:38,643 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:38,643 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:38,643 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:38,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:38,645 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,646 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786518327_0001_1_00_000000_0_10047, runDuration: 54999, ]
2016-01-14 15:48:38,648 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786518327_0001_1_00_000000_0_10047] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786518648, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:38,648 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:38,648 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,649 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 2ED7AC534669403FB9B25EDE2400DAD6
2016-01-14 15:48:38,649 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"], id: 9DC90CBA103E4575BC5169E30B287DA7
2016-01-14 15:48:38,649 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"]
2016-01-14 15:48:38,649 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 2ED7AC534669403FB9B25EDE2400DAD6, for 1 inputs
2016-01-14 15:48:38,649 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:38,650 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [EF1ACD2574CE4C67966F6DB70D92535D] #9 in 3ms
2016-01-14 15:48:38,651 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EF1ACD2574CE4C67966F6DB70D92535D] #9, status:false, isInterrupted:false
2016-01-14 15:48:38,651 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:38,652 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:38,652 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:38,652 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:38,652 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:38,652 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,652 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:38,652 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B720CBD51C0C48DF97E86274C0E700E4, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:48:38,653 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@22d642b0; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@20b442ae
2016-01-14 15:48:38,658 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:38,659 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:38,659 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,659 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786518327_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:38,660 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,660 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B720CBD51C0C48DF97E86274C0E700E4, mem on close (mb), free: 192, total: 432, max: 672
2016-01-14 15:48:38,660 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,661 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:38,661 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:38,661 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:38,661 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:38,661 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:38 Completed running task attempt: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,662 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:38,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0, startTime=1452786518619, finishTime=1452786518663, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518653, Process_Duration=7, Process_End_Time=1452786518660, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:38,663 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:38,664 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:38,664 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,664 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:38,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_FINISHED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, taskId=task_1452786518327_0001_1_01_000000, startTime=1452786518619, finishTime=1452786518664, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452786518327_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518653, Process_Duration=7, Process_End_Time=1452786518660, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:38,665 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:38,665 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] : 1
2016-01-14 15:48:38,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:38,665 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_FINISHED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, vertexId=vertex_1452786518327_0001_1_01, initRequestedTime=1452786518510, initedTime=1452786518520, startRequestedTime=1452786518526, startedTime=1452786518526, finishTime=1452786518665, timeTaken=139, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518653, Process_Duration=7, Process_End_Time=1452786518660, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786518619, firstTasksToStart=[ task_1452786518327_0001_1_01_000000 ], lastTaskFinishTime=1452786518663, lastTasksToFinish=[ task_1452786518327_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786518327_0001_1_01_000000 ], longestDurationTasks=[ task_1452786518327_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:38,667 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:38,667 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:38,668 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786518327_0001_00_000002, stoppedTime=1452786518668, exitStatus=0
2016-01-14 15:48:38,668 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:38,669 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:38,669 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786518327_0001_1
2016-01-14 15:48:38,669 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786518327_0001_1
2016-01-14 15:48:38,670 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,670 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9DC90CBA103E4575BC5169E30B287DA7 for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:38,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786518653, Process_Duration=7, Process_End_Time=1452786518660, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:38,671 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:38,671 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,672 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518327_0001_00_000002
2016-01-14 15:48:38,672 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786518327_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:38,672 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518327_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:38,672 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786518327_0001_00_000002, Exiting
2016-01-14 15:48:38,672 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786518327_0001_00_000002
2016-01-14 15:48:38,672 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786518327_0001_00_000002
2016-01-14 15:48:38,673 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786518327_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:38,673 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786518327_0001_00_000002 completed successfully
2016-01-14 15:48:38,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:38,681 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786518327_0001_1, queueSize=0
2016-01-14 15:48:38,682 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:38,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786518327_0001_1, startTime=1452786518507, finishTime=1452786518680, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=905969664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573037241, Process_Duration=11, Process_End_Time=2905573037252, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:38,682 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786518327_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:38,683 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518327_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:38,683 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786518327_0001_00_000002
2016-01-14 15:48:38,683 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:38 Completed Dag: dag_1452786518327_0001_1
2016-01-14 15:48:38,684 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786518327_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:38,684 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786518327_0001_1_post
2016-01-14 15:48:38,685 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,685 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:38,688 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1452786518327_0001
2016-01-14 15:48:38,692 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:38,692 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:38,693 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:38,693 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:38,693 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:38,693 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:38,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:38,694 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:38,694 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:38,694 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:38,694 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:38,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:38,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:38,700 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001
2016-01-14 15:48:38,701 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.391
2016-01-14 15:48:38,760 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:38,760 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:38,793 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 15:48:38,809 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-01-14 15:48:38,809 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:38,810 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:38,810 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:38,829 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:38,841 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false
2016-01-14 15:48:38,898 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:38,898 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:38,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"]
2016-01-14 15:48:38,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:38,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:38,900 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:38,900 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ve/true_false_false_false
2016-01-14 15:48:38,901 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:38,909 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:38,913 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:38,914 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:38,914 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:38,923 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001 doesn't exist and is created
2016-01-14 15:48:38,939 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:38,939 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:38,940 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001
2016-01-14 15:48:38,943 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786518914_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:38,950 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:38,951 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:38,953 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:38,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:38,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001/localmode-log-dir
2016-01-14 15:48:38,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001/localmode-log-dir/history.txt.appattempt_1452786518914_0001_000000, maxErrors=10
2016-01-14 15:48:38,954 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:38,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786518914_0001_000000, appSubmitTime=1452786518942, launchTime=1452786518942
2016-01-14 15:48:38,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:38,959 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:38,960 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:38,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:38,963 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39006
2016-01-14 15:48:38,965 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:38,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786518914_0001_000000, startTime=1452786518968
2016-01-14 15:48:38,968 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:38,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:39,040 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:39,040 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:39,040 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1452786518914_0001, dagName=[DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-01-14 15:48:39,056 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-01-14 15:48:39,057 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786518914_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001/localmode-log-dir/dag_1452786518914_0001_1.dot
2016-01-14 15:48:39,058 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001/localmode-log-dir/dag_1452786518914_0001_1-tez-dag.pb.txt
2016-01-14 15:48:39,069 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786518914_0001_1
2016-01-14 15:48:39,069 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:39,069 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-01-14 15:48:39 Running Dag: dag_1452786518914_0001_1
2016-01-14 15:48:39,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786518914_0001_1, submitTime=1452786519056
2016-01-14 15:48:39,084 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:39,085 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,085 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:39,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786518914_0001_1, initTime=1452786519084
2016-01-14 15:48:39,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:39,086 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518914_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:39,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_STARTED]: dagID=dag_1452786518914_0001_1, startTime=1452786519086
2016-01-14 15:48:39,086 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:39,087 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518914_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:39,087 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E4B20FBCBCE94A4EA6E4C85655610121 : {9806DA84762545DC85FAC9EACDBCED3B={InputName=9806DA84762545DC85FAC9EACDBCED3B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:39,087 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9806DA84762545DC85FAC9EACDBCED3B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:39,087 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:39,087 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,088 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,088 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]. Starting root input initializers: 1
2016-01-14 15:48:39,088 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:39,089 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,089 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9806DA84762545DC85FAC9EACDBCED3B on vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,091 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:39,091 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9806DA84762545DC85FAC9EACDBCED3B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:39,091 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:39,092 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:39,094 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:39,094 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9806DA84762545DC85FAC9EACDBCED3B on vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,097 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:39,097 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,097 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,098 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,098 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D392D21B56624D1F8E3194920657D6D3, vertexId=vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:39,098 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D392D21B56624D1F8E3194920657D6D3, vertexId=vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,101 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1452786518914_0001, dagName=[DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-01-14 15:48:39,102 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D392D21B56624D1F8E3194920657D6D3, vertexName=C43E0A9295CB4D9B8303357E015B7140, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:39,103 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D392D21B56624D1F8E3194920657D6D3, vertexId=vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:39,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, vertexId=vertex_1452786518914_0001_1_01, initRequestedTime=1452786519089, initedTime=1452786519104, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:39,106 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:39,106 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:39,106 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9806DA84762545DC85FAC9EACDBCED3B=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:39,107 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] parallelism set to 1
2016-01-14 15:48:39,107 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:39,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786518914_0001_1_00, eventCount=1
2016-01-14 15:48:39,109 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:39,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, vertexId=vertex_1452786518914_0001_1_00, initRequestedTime=1452786519087, initedTime=1452786519109, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:39,110 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] with distanceFromRoot: 0
2016-01-14 15:48:39,111 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:39,111 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:39,112 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:39,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786518914_0001_1_00, startRequestedTime=1452786519111, startedTime=1452786519111
2016-01-14 15:48:39,113 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:39,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:39,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_STARTED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, taskId=task_1452786518914_0001_1_00_000000, scheduledTime=1452786519114, launchTime=1452786519114
2016-01-14 15:48:39,115 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:39,115 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786518914_0001_1_00 for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] numStartedSources: 1 numSources: 1
2016-01-14 15:48:39,116 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] with distanceFromRoot: 1
2016-01-14 15:48:39,116 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,117 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E4B20FBCBCE94A4EA6E4C85655610121 in vertex: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,117 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C43E0A9295CB4D9B8303357E015B7140 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:39,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:39,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786518914_0001_1_01, startRequestedTime=1452786519117, startedTime=1452786519117
2016-01-14 15:48:39,118 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:39,119 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: E4B20FBCBCE94A4EA6E4C85655610121, VertexParallelism: 1, TaskAttemptID:attempt_1452786518914_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9806DA84762545DC85FAC9EACDBCED3B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C43E0A9295CB4D9B8303357E015B7140, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:39,119 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:39,119 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:39,120 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:39,120 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:39,120 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:39,120 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786518914_0001, with dag id: dag_1452786518914_0001_1
2016-01-14 15:48:39,121 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786518914_0001_00_000001
2016-01-14 15:48:39,122 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:39,124 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786518914_0001_00_000001
2016-01-14 15:48:39,124 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:39,125 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:39,125 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:39,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786518914_0001_00_000001, launchTime=1452786519125
2016-01-14 15:48:39,127 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518914_0001_00_000001
2016-01-14 15:48:39,127 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786518914_0001_1_00_000000_0] to container: [container_1452786518914_0001_00_000001]
2016-01-14 15:48:39,127 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:39,128 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786518914_0001_00_000001 given task: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,128 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518914_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,128 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786518914_0001_1_00_000000_0] started. Is using containerId: [container_1452786518914_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:39,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0, startTime=1452786519128, containerId=container_1452786518914_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786518914_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:39,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:39,129 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:39,129 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:39 Starting to run new task attempt: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,129 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,129 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:39,130 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:39,130 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:39,130 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:39,130 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:39,130 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:39,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: E4B20FBCBCE94A4EA6E4C85655610121, VertexParallelism: 1, TaskAttemptID:attempt_1452786518914_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9806DA84762545DC85FAC9EACDBCED3B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C43E0A9295CB4D9B8303357E015B7140, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:39,132 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:39,133 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:39,133 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786518914_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:39,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:39,134 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9806DA84762545DC85FAC9EACDBCED3B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:39,134 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C43E0A9295CB4D9B8303357E015B7140, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:39,134 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:39,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:39,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:39,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,136 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,140 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:39,141 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:39,141 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,140 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:39,143 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,148 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4B20FBCBCE94A4EA6E4C85655610121, ordinal: 0
2016-01-14 15:48:39,148 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4B20FBCBCE94A4EA6E4C85655610121, mem on start (mb), free: 105, total: 432, max: 672
2016-01-14 15:48:39,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:39,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:39,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:39,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:39,148 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:39,148 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:39,149 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:39,149 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:39,149 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:39,149 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:39,150 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9806DA84762545DC85FAC9EACDBCED3B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:39,150 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C43E0A9295CB4D9B8303357E015B7140, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:39,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:39,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9806DA84762545DC85FAC9EACDBCED3B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:39,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:39,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:39,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,151 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:39,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:39,152 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,155 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:39,156 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,156 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 1A8E1308617942248E56FF4F126F68B8
2016-01-14 15:48:39,157 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 1A8E1308617942248E56FF4F126F68B8
2016-01-14 15:48:39,157 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:39,158 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5a3c86c8
2016-01-14 15:48:39,158 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5a3c86c8
2016-01-14 15:48:39,159 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:39,159 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:39,160 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@3bb4e2c0; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6092c9df; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@45038101; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:39,160 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:39,161 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:39,161 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:39,183 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:39,184 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:39,184 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:39,184 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:39,184 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:39,185 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:39,185 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:39,185 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4B20FBCBCE94A4EA6E4C85655610121, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:39,189 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,190 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4B20FBCBCE94A4EA6E4C85655610121, mem on close (mb), free: 192, total: 422, max: 672
2016-01-14 15:48:39,190 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:39,190 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:39,190 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:39,190 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:40,327 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:40,327 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:40,896 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:40,896 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:41,550 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:41 Completed running task attempt: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:41,553 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:41,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786519185, Process_Duration=4, Process_End_Time=1452786519189, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:41,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786518914_0001_1_00, eventCount=1
2016-01-14 15:48:41,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0, startTime=1452786519128, finishTime=1452786521554, timeTaken=2426, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786519185, Process_Duration=4, Process_End_Time=1452786519189, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:41,554 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:41,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:41,554 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:41,555 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:41,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_FINISHED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, taskId=task_1452786518914_0001_1_00_000000, startTime=1452786519128, finishTime=1452786521554, timeTaken=2426, status=SUCCEEDED, successfulAttemptID=attempt_1452786518914_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786519185, Process_Duration=4, Process_End_Time=1452786519189, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:41,555 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:41,555 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:41,555 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] : 1
2016-01-14 15:48:41,555 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:41,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_FINISHED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, vertexId=vertex_1452786518914_0001_1_00, initRequestedTime=1452786519087, initedTime=1452786519109, startRequestedTime=1452786519111, startedTime=1452786519111, finishTime=1452786521555, timeTaken=2444, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786519185, Process_Duration=4, Process_End_Time=1452786519189, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786519128, firstTasksToStart=[ task_1452786518914_0001_1_00_000000 ], lastTaskFinishTime=1452786521554, lastTasksToFinish=[ task_1452786518914_0001_1_00_000000 ], minTaskDuration=2426, maxTaskDuration=2426, avgTaskDuration=2426.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786518914_0001_1_00_000000 ], longestDurationTasks=[ task_1452786518914_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:41,557 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:41,557 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:41,557 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] attempt: attempt_1452786518914_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:41,557 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:41,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:41,557 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:41,557 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,558 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:41,558 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:41,558 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786518914_0001_00_000001, stoppedTime=1452786521558, exitStatus=0
2016-01-14 15:48:41,559 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:41,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_STARTED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, taskId=task_1452786518914_0001_1_01_000000, scheduledTime=1452786521559, launchTime=1452786521559
2016-01-14 15:48:41,559 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:41,559 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:41,559 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786518914_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:41,559 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: C43E0A9295CB4D9B8303357E015B7140, VertexParallelism: 1, TaskAttemptID:attempt_1452786518914_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4B20FBCBCE94A4EA6E4C85655610121, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D392D21B56624D1F8E3194920657D6D3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:41,559 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:41,560 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:41,560 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:41,560 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:41,558 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518914_0001_00_000001
2016-01-14 15:48:41,560 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:41,561 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786518914_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:41,561 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518914_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:41,561 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786518914_0001_00_000002
2016-01-14 15:48:41,562 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786518914_0001_00_000001, Exiting
2016-01-14 15:48:41,562 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786518914_0001_00_000001
2016-01-14 15:48:41,562 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786518914_0001_00_000001
2016-01-14 15:48:41,562 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:41,564 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786518914_0001_00_000002
2016-01-14 15:48:41,564 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786518914_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:41,564 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786518914_0001_00_000001 completed successfully
2016-01-14 15:48:41,565 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:41,565 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:41,567 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:41,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786518914_0001_00_000002, launchTime=1452786521567
2016-01-14 15:48:41,567 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:41,568 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518914_0001_00_000002
2016-01-14 15:48:41,568 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786518914_0001_1_01_000000_0] to container: [container_1452786518914_0001_00_000002]
2016-01-14 15:48:41,568 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:41,569 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786518914_0001_00_000002 given task: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,569 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518914_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,569 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786518914_0001_1_01_000000_0] started. Is using containerId: [container_1452786518914_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:41 Starting to run new task attempt: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,569 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,569 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,569 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:41,570 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:41,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:41,570 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:41,570 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:41,570 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:41,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0, startTime=1452786521569, containerId=container_1452786518914_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786518914_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:41,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: C43E0A9295CB4D9B8303357E015B7140, VertexParallelism: 1, TaskAttemptID:attempt_1452786518914_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4B20FBCBCE94A4EA6E4C85655610121, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D392D21B56624D1F8E3194920657D6D3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:41,570 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:41,571 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:41,571 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:41,571 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:41,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:41,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:41,573 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786518914_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:41,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D392D21B56624D1F8E3194920657D6D3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:41,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E4B20FBCBCE94A4EA6E4C85655610121, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:41,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:41,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D392D21B56624D1F8E3194920657D6D3
2016-01-14 15:48:41,575 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:41,575 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D392D21B56624D1F8E3194920657D6D3
2016-01-14 15:48:41,576 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:41,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:41,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,579 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:41,579 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:41,579 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,584 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C43E0A9295CB4D9B8303357E015B7140, ordinal: 1
2016-01-14 15:48:41,584 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C43E0A9295CB4D9B8303357E015B7140, mem on start (mb), free: 230, total: 444, max: 672
2016-01-14 15:48:41,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:41,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:41,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:41,589 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:41,589 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D392D21B56624D1F8E3194920657D6D3
2016-01-14 15:48:41,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:41,590 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:41,590 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:41,590 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:41,590 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:41,590 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:41,591 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:41,591 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D392D21B56624D1F8E3194920657D6D3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:41,591 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E4B20FBCBCE94A4EA6E4C85655610121, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:41,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:41,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E4B20FBCBCE94A4EA6E4C85655610121 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:41,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:41,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:41,593 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:41,593 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:41,593 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E4B20FBCBCE94A4EA6E4C85655610121 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:41,594 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:41,594 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:41,594 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:41,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:41,595 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,598 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:41,598 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786518914_0001_1_00_000000_0_10053, runDuration: 2415393, ]
2016-01-14 15:48:41,598 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 1A8E1308617942248E56FF4F126F68B8
2016-01-14 15:48:41,599 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"], id: D392D21B56624D1F8E3194920657D6D3
2016-01-14 15:48:41,599 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"]
2016-01-14 15:48:41,599 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 1A8E1308617942248E56FF4F126F68B8, for 1 inputs
2016-01-14 15:48:41,599 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786518914_0001_1_00_000000_0_10053] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786521599, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 15:48:41,600 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,600 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:41,600 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E4B20FBCBCE94A4EA6E4C85655610121] #10 in 1ms
2016-01-14 15:48:41,600 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E4B20FBCBCE94A4EA6E4C85655610121] #10, status:false, isInterrupted:false
2016-01-14 15:48:41,600 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:41,601 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:41,601 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:41,601 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:41,601 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:41,601 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,602 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:41,602 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C43E0A9295CB4D9B8303357E015B7140, all 1 inputs ready in: 00:00:00.002
2016-01-14 15:48:41,603 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@7b414197; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@79395fb8
2016-01-14 15:48:41,607 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:41,607 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:41,607 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,608 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786518914_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:41,608 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,608 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C43E0A9295CB4D9B8303357E015B7140, mem on close (mb), free: 226, total: 444, max: 672
2016-01-14 15:48:41,609 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,609 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:41,609 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:41,610 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:41,610 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:41,610 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:41 Completed running task attempt: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,611 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:41,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0, startTime=1452786521569, finishTime=1452786521612, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786521602, Process_Duration=6, Process_End_Time=1452786521608, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:41,612 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:41,612 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:41,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_FINISHED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, taskId=task_1452786518914_0001_1_01_000000, startTime=1452786521569, finishTime=1452786521612, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452786518914_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786521602, Process_Duration=6, Process_End_Time=1452786521608, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:41,613 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:41,613 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] : 1
2016-01-14 15:48:41,613 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:41,613 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:41,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:41,614 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:41,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_FINISHED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, vertexId=vertex_1452786518914_0001_1_01, initRequestedTime=1452786519089, initedTime=1452786519104, startRequestedTime=1452786519117, startedTime=1452786519117, finishTime=1452786521613, timeTaken=2496, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786521602, Process_Duration=6, Process_End_Time=1452786521608, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786521569, firstTasksToStart=[ task_1452786518914_0001_1_01_000000 ], lastTaskFinishTime=1452786521612, lastTasksToFinish=[ task_1452786518914_0001_1_01_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786518914_0001_1_01_000000 ], longestDurationTasks=[ task_1452786518914_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:41,614 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:41,615 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:41,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:41,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786518914_0001_00_000002, stoppedTime=1452786521615, exitStatus=0
2016-01-14 15:48:41,615 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:41,615 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:41,615 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786518914_0001_1
2016-01-14 15:48:41,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786518914_0001_1
2016-01-14 15:48:41,616 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:41,616 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D392D21B56624D1F8E3194920657D6D3 for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:41,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786521602, Process_Duration=6, Process_End_Time=1452786521608, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:41,619 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:41,619 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,620 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518914_0001_00_000002
2016-01-14 15:48:41,620 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786518914_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:41,620 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518914_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:41,621 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786518914_0001_00_000002, Exiting
2016-01-14 15:48:41,621 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786518914_0001_00_000002
2016-01-14 15:48:41,621 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786518914_0001_00_000002
2016-01-14 15:48:41,623 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786518914_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:41,624 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786518914_0001_00_000002 completed successfully
2016-01-14 15:48:41,630 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786518914_0001_1, queueSize=0
2016-01-14 15:48:41,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:41,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786518914_0001_1, startTime=1452786519086, finishTime=1452786521628, timeTaken=2542, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=1518, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=931135488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573040787, Process_Duration=10, Process_End_Time=2905573040797, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:41,630 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786518914_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:41,631 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518914_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:41,631 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:41,631 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786518914_0001_00_000002
2016-01-14 15:48:41 Completed Dag: dag_1452786518914_0001_1
2016-01-14 15:48:41,632 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786518914_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:41,632 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786518914_0001_1_post
2016-01-14 15:48:41,633 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,633 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:41,636 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1452786518914_0001
2016-01-14 15:48:41,638 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:41,638 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:41,638 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:41,638 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:41,639 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:41,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:41,642 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:41,642 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:41,642 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:41,643 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:41,643 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:41,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:41,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:41,649 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001
2016-01-14 15:48:41,649 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:02.751
2016-01-14 15:48:41,703 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:41,703 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:41,742 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:48:41,750 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-01-14 15:48:41,750 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:41,751 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:41,751 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:41,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:41,770 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false
2016-01-14 15:48:41,806 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:41,807 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:41,807 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"]
2016-01-14 15:48:41,807 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:41,807 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:41,807 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:41,808 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/true_true_false_false
2016-01-14 15:48:41,808 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:41,817 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:41,821 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:41,822 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:41,822 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:41,831 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001 doesn't exist and is created
2016-01-14 15:48:41,846 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:41,846 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:41,847 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001
2016-01-14 15:48:41,850 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786521822_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:41,857 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:41,858 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:41,860 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:41,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:41,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001/localmode-log-dir
2016-01-14 15:48:41,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001/localmode-log-dir/history.txt.appattempt_1452786521822_0001_000000, maxErrors=10
2016-01-14 15:48:41,861 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:41,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786521822_0001_000000, appSubmitTime=1452786521850, launchTime=1452786521850
2016-01-14 15:48:41,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:41,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:41,866 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:41,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:41,869 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53399
2016-01-14 15:48:41,870 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:41,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786521822_0001_000000, startTime=1452786521872
2016-01-14 15:48:41,872 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:41,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:41,947 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:41,947 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:41,947 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1452786521822_0001, dagName=[8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-01-14 15:48:41,960 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-01-14 15:48:41,961 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786521822_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001/localmode-log-dir/dag_1452786521822_0001_1.dot
2016-01-14 15:48:41,962 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001/localmode-log-dir/dag_1452786521822_0001_1-tez-dag.pb.txt
2016-01-14 15:48:41,970 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786521822_0001_1
2016-01-14 15:48:41,970 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,970 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-01-14 15:48:41 Running Dag: dag_1452786521822_0001_1
2016-01-14 15:48:41,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786521822_0001_1, submitTime=1452786521960
2016-01-14 15:48:41,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:41,983 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:41,984 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:41,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786521822_0001_1, initTime=1452786521983
2016-01-14 15:48:41,985 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786521822_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:41,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:41,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_STARTED]: dagID=dag_1452786521822_0001_1, startTime=1452786521985
2016-01-14 15:48:41,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:41,986 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:41,986 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786521822_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:41,986 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FF4F13E6FC2547F9A580FDA4118AB231 : {22A682D8507F41DA9000547AFD90768B={InputName=22A682D8507F41DA9000547AFD90768B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:41,986 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1452786521822_0001, dagName=[8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-01-14 15:48:41,986 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 22A682D8507F41DA9000547AFD90768B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:41,987 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,987 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,987 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,987 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]. Starting root input initializers: 1
2016-01-14 15:48:41,988 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:41,988 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,988 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 22A682D8507F41DA9000547AFD90768B on vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,990 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:41,990 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 22A682D8507F41DA9000547AFD90768B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:41,990 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:41,991 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:41,991 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:41,992 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,992 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,992 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,992 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=57285A20841E45FAB7A6A61E184394DF, vertexId=vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:41,993 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=57285A20841E45FAB7A6A61E184394DF, vertexId=vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,994 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:41,994 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 22A682D8507F41DA9000547AFD90768B on vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,995 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=57285A20841E45FAB7A6A61E184394DF, vertexName=BF55D45B5B2C43B7A40673361A13E1FC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:41,995 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=57285A20841E45FAB7A6A61E184394DF, vertexId=vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,996 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:41,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, vertexId=vertex_1452786521822_0001_1_01, initRequestedTime=1452786521988, initedTime=1452786521996, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:41,997 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:41,997 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:41,997 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {22A682D8507F41DA9000547AFD90768B=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:41,998 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] parallelism set to 1
2016-01-14 15:48:41,998 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:41,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786521822_0001_1_00, eventCount=1
2016-01-14 15:48:41,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, vertexId=vertex_1452786521822_0001_1_00, initRequestedTime=1452786521986, initedTime=1452786521999, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:41,999 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] with distanceFromRoot: 0
2016-01-14 15:48:41,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:41,999 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:42,000 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:42,000 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:42,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:42,000 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786521822_0001, with dag id: dag_1452786521822_0001_1
2016-01-14 15:48:42,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786521822_0001_1_00, startRequestedTime=1452786522000, startedTime=1452786522000
2016-01-14 15:48:42,002 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:42,002 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:42,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_STARTED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, taskId=task_1452786521822_0001_1_00_000000, scheduledTime=1452786522002, launchTime=1452786522002
2016-01-14 15:48:42,003 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:42,003 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786521822_0001_1_00 for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] numStartedSources: 1 numSources: 1
2016-01-14 15:48:42,003 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] with distanceFromRoot: 1
2016-01-14 15:48:42,004 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:42,004 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FF4F13E6FC2547F9A580FDA4118AB231 in vertex: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:42,004 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BF55D45B5B2C43B7A40673361A13E1FC with 1 source tasks and 1 pending tasks
2016-01-14 15:48:42,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:42,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786521822_0001_1_01, startRequestedTime=1452786522004, startedTime=1452786522004
2016-01-14 15:48:42,005 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:42,006 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: FF4F13E6FC2547F9A580FDA4118AB231, VertexParallelism: 1, TaskAttemptID:attempt_1452786521822_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=22A682D8507F41DA9000547AFD90768B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BF55D45B5B2C43B7A40673361A13E1FC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:42,007 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:42,007 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:42,009 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:42,009 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:42,010 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:42,010 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786521822_0001_00_000001
2016-01-14 15:48:42,010 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:42,012 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786521822_0001_00_000001
2016-01-14 15:48:42,012 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:42,012 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:42,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:42,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786521822_0001_00_000001, launchTime=1452786522013
2016-01-14 15:48:42,014 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786521822_0001_00_000001
2016-01-14 15:48:42,014 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786521822_0001_1_00_000000_0] to container: [container_1452786521822_0001_00_000001]
2016-01-14 15:48:42,015 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:42,015 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786521822_0001_00_000001 given task: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,015 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786521822_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42 Starting to run new task attempt: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,015 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786521822_0001_1_00_000000_0] started. Is using containerId: [container_1452786521822_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:42,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0, startTime=1452786522015, containerId=container_1452786521822_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786521822_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:42,016 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:42,016 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:42,016 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:42,016 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:42,017 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:42,017 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:42,017 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:42,017 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:42,018 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:42,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: FF4F13E6FC2547F9A580FDA4118AB231, VertexParallelism: 1, TaskAttemptID:attempt_1452786521822_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=22A682D8507F41DA9000547AFD90768B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BF55D45B5B2C43B7A40673361A13E1FC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:42,018 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:42,018 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:42,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:42,019 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786521822_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:42,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=22A682D8507F41DA9000547AFD90768B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:42,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BF55D45B5B2C43B7A40673361A13E1FC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:42,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:42,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:42,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,021 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,023 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:42,023 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:42,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:42,026 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:42,026 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:42,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:42,027 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:42,034 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF4F13E6FC2547F9A580FDA4118AB231, ordinal: 0
2016-01-14 15:48:42,034 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF4F13E6FC2547F9A580FDA4118AB231, mem on start (mb), free: 137, total: 444, max: 672
2016-01-14 15:48:42,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:42,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:42,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:42,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:42,035 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:42,035 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:42,035 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:42,035 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:42,036 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:42,036 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:42,036 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 22A682D8507F41DA9000547AFD90768B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:42,036 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BF55D45B5B2C43B7A40673361A13E1FC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:42,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:42,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,037 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:42,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 22A682D8507F41DA9000547AFD90768B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:42,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:42,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:42,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:42,038 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,038 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:42,038 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@19ede0f3
2016-01-14 15:48:42,039 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@19ede0f3
2016-01-14 15:48:42,040 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:42,040 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:42,041 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:42,042 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,042 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 37048EF07AFD4E7090B99EC02EB63F37
2016-01-14 15:48:42,042 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 37048EF07AFD4E7090B99EC02EB63F37
2016-01-14 15:48:42,043 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@2fd492e8; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2cf8d5e0; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@4a17c31; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:42,044 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:42,044 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:42,045 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:42,064 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:42,064 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:42,064 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:42,064 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:42,065 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:42,065 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:42,065 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:42,066 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF4F13E6FC2547F9A580FDA4118AB231, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:42,070 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,070 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF4F13E6FC2547F9A580FDA4118AB231, mem on close (mb), free: 207, total: 444, max: 672
2016-01-14 15:48:42,070 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:42,070 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:42,070 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:42,071 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:43,134 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:43,135 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:43,693 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:43,694 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:44,967 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:44 Completed running task attempt: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:44,972 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:44,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786521822_0001_1_00, eventCount=1
2016-01-14 15:48:44,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786522066, Process_Duration=4, Process_End_Time=1452786522070, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:44,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0, startTime=1452786522015, finishTime=1452786524973, timeTaken=2958, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786522066, Process_Duration=4, Process_End_Time=1452786522070, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:44,973 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:44,974 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:44,973 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:44,974 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:44,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_FINISHED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, taskId=task_1452786521822_0001_1_00_000000, startTime=1452786522015, finishTime=1452786524974, timeTaken=2959, status=SUCCEEDED, successfulAttemptID=attempt_1452786521822_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786522066, Process_Duration=4, Process_End_Time=1452786522070, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:44,974 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:44,975 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786521822_0001_00_000001
2016-01-14 15:48:44,975 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:44,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:44,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:44,975 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] : 1
2016-01-14 15:48:44,976 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:44,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_FINISHED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, vertexId=vertex_1452786521822_0001_1_00, initRequestedTime=1452786521986, initedTime=1452786521999, startRequestedTime=1452786522000, startedTime=1452786522000, finishTime=1452786524976, timeTaken=2976, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786522066, Process_Duration=4, Process_End_Time=1452786522070, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786522015, firstTasksToStart=[ task_1452786521822_0001_1_00_000000 ], lastTaskFinishTime=1452786524973, lastTasksToFinish=[ task_1452786521822_0001_1_00_000000 ], minTaskDuration=2958, maxTaskDuration=2958, avgTaskDuration=2958.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786521822_0001_1_00_000000 ], longestDurationTasks=[ task_1452786521822_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:44,977 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:44,977 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:44,977 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] attempt: attempt_1452786521822_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:44,978 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:44,978 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:44,978 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:44,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786521822_0001_00_000001, stoppedTime=1452786524978, exitStatus=0
2016-01-14 15:48:44,978 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:44,978 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:44,978 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:44,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_STARTED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, taskId=task_1452786521822_0001_1_01_000000, scheduledTime=1452786524979, launchTime=1452786524979
2016-01-14 15:48:44,979 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:44,979 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:44,980 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786521822_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:44,980 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:44,981 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:44,981 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: BF55D45B5B2C43B7A40673361A13E1FC, VertexParallelism: 1, TaskAttemptID:attempt_1452786521822_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF4F13E6FC2547F9A580FDA4118AB231, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=57285A20841E45FAB7A6A61E184394DF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:44,981 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:44,981 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:44,982 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786521822_0001_00_000002
2016-01-14 15:48:44,982 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:44,983 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786521822_0001_00_000002
2016-01-14 15:48:44,983 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:44,984 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:44,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:44,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786521822_0001_00_000002, launchTime=1452786524984
2016-01-14 15:48:44,985 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786521822_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:44,985 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786521822_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:44,985 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786521822_0001_00_000001, Exiting
2016-01-14 15:48:44,985 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786521822_0001_00_000001
2016-01-14 15:48:44,985 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786521822_0001_00_000001
2016-01-14 15:48:44,987 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786521822_0001_00_000002
2016-01-14 15:48:44,988 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786521822_0001_1_01_000000_0] to container: [container_1452786521822_0001_00_000002]
2016-01-14 15:48:44,988 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:44,988 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786521822_0001_00_000002 given task: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,989 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786521822_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786521822_0001_1_01_000000_0] started. Is using containerId: [container_1452786521822_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:44,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0, startTime=1452786524989, containerId=container_1452786521822_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786521822_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:44,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:44,989 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:44 Starting to run new task attempt: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,989 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,990 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:44,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:44,990 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786521822_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:44,990 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:44,990 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786521822_0001_00_000001 completed successfully
2016-01-14 15:48:44,991 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:44,991 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:44,992 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:44,992 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:44,992 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:44,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: BF55D45B5B2C43B7A40673361A13E1FC, VertexParallelism: 1, TaskAttemptID:attempt_1452786521822_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF4F13E6FC2547F9A580FDA4118AB231, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=57285A20841E45FAB7A6A61E184394DF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:44,993 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:44,993 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:44,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:44,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FF4F13E6FC2547F9A580FDA4118AB231, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:44,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=57285A20841E45FAB7A6A61E184394DF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:44,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:44,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:44,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:44,997 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:44,995 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786521822_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:44,998 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:44,998 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:44,998 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:44,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:44,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:44,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 57285A20841E45FAB7A6A61E184394DF
2016-01-14 15:48:45,000 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:45,000 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 57285A20841E45FAB7A6A61E184394DF
2016-01-14 15:48:45,006 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BF55D45B5B2C43B7A40673361A13E1FC, ordinal: 1
2016-01-14 15:48:45,006 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF55D45B5B2C43B7A40673361A13E1FC, mem on start (mb), free: 193, total: 447, max: 672
2016-01-14 15:48:45,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:45,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:45,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:45,012 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:45,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 57285A20841E45FAB7A6A61E184394DF
2016-01-14 15:48:45,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:45,012 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:45,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:45,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:45,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:45,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:45,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:45,013 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FF4F13E6FC2547F9A580FDA4118AB231, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:45,013 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 57285A20841E45FAB7A6A61E184394DF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:45,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:45,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FF4F13E6FC2547F9A580FDA4118AB231 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:45,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:45,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:45,015 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:45,015 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:45,016 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FF4F13E6FC2547F9A580FDA4118AB231 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:45,016 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:45,016 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:45,016 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:45,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:45,019 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,020 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786521822_0001_1_00_000000_0_10059, runDuration: 2945228, ]
2016-01-14 15:48:45,022 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:45,022 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786521822_0001_1_00_000000_0_10059] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786525022, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:45,022 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 37048EF07AFD4E7090B99EC02EB63F37
2016-01-14 15:48:45,022 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"], id: 57285A20841E45FAB7A6A61E184394DF
2016-01-14 15:48:45,022 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,023 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"]
2016-01-14 15:48:45,023 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 37048EF07AFD4E7090B99EC02EB63F37, for 1 inputs
2016-01-14 15:48:45,023 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:45,024 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FF4F13E6FC2547F9A580FDA4118AB231] #11 in 3ms
2016-01-14 15:48:45,024 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FF4F13E6FC2547F9A580FDA4118AB231] #11, status:false, isInterrupted:false
2016-01-14 15:48:45,025 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:45,026 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:45,026 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:45,026 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:45,027 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:45,027 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,027 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BF55D45B5B2C43B7A40673361A13E1FC, all 1 inputs ready in: 00:00:00.004
2016-01-14 15:48:45,027 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:45,028 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@15646798; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@72cd3720
2016-01-14 15:48:45,032 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:45,032 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:45,032 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,033 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786521822_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:45,033 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,033 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF55D45B5B2C43B7A40673361A13E1FC, mem on close (mb), free: 189, total: 447, max: 672
2016-01-14 15:48:45,034 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,034 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:45,035 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:45,035 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:45,035 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:45,035 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:45 Completed running task attempt: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,036 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:45,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0, startTime=1452786524989, finishTime=1452786525037, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525027, Process_Duration=6, Process_End_Time=1452786525033, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:45,037 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:45,038 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:45,038 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:45,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_FINISHED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, taskId=task_1452786521822_0001_1_01_000000, startTime=1452786524989, finishTime=1452786525038, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452786521822_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525027, Process_Duration=6, Process_End_Time=1452786525033, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:45,038 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:45,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:45,039 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] : 1
2016-01-14 15:48:45,039 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:45,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:45,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_FINISHED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, vertexId=vertex_1452786521822_0001_1_01, initRequestedTime=1452786521988, initedTime=1452786521996, startRequestedTime=1452786522004, startedTime=1452786522004, finishTime=1452786525039, timeTaken=3035, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525027, Process_Duration=6, Process_End_Time=1452786525033, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786524989, firstTasksToStart=[ task_1452786521822_0001_1_01_000000 ], lastTaskFinishTime=1452786525037, lastTasksToFinish=[ task_1452786521822_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786521822_0001_1_01_000000 ], longestDurationTasks=[ task_1452786521822_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:45,040 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:45,040 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:45,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786521822_0001_00_000002, stoppedTime=1452786525040, exitStatus=0
2016-01-14 15:48:45,041 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:45,041 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:45,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:45,041 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786521822_0001_1
2016-01-14 15:48:45,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786521822_0001_1
2016-01-14 15:48:45,041 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 57285A20841E45FAB7A6A61E184394DF for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:45,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786525027, Process_Duration=6, Process_End_Time=1452786525033, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:45,045 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:45,046 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:45,046 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786521822_0001_00_000002
2016-01-14 15:48:45,048 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786521822_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:45,048 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786521822_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:45,049 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786521822_0001_00_000002, Exiting
2016-01-14 15:48:45,049 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786521822_0001_00_000002
2016-01-14 15:48:45,049 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786521822_0001_00_000002
2016-01-14 15:48:45,049 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786521822_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:45,049 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786521822_0001_00_000002 completed successfully
2016-01-14 15:48:45,053 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:45,055 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786521822_0001_1, queueSize=0
2016-01-14 15:48:45,055 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:45,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786521822_0001_1, startTime=1452786521985, finishTime=1452786525053, timeTaken=3068, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=938475520, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573047093, Process_Duration=10, Process_End_Time=2905573047103, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:45,055 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786521822_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:45,055 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786521822_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:45,055 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:45 Completed Dag: dag_1452786521822_0001_1
2016-01-14 15:48:45,056 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786521822_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:45,057 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786521822_0001_1_post
2016-01-14 15:48:45,057 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:45,057 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:45,058 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786521822_0001_00_000002
2016-01-14 15:48:45,061 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1452786521822_0001
2016-01-14 15:48:45,063 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:45,063 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:45,063 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:45,063 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:45,063 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:45,063 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:45,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:45,065 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:45,065 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:45,065 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:45,066 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:45,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:45,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:45,072 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001
2016-01-14 15:48:45,072 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:03.266
2016-01-14 15:48:45,128 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:45,129 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:45,165 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 15:48:45,170 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-01-14 15:48:45,170 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:45,170 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:45,170 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:45,182 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:45,190 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false
2016-01-14 15:48:45,227 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:45,228 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:45,228 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false"]
2016-01-14 15:48:45,228 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:45,228 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:45,228 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:45,230 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ve/false_true_false_false
2016-01-14 15:48:45,230 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:45,240 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:45,244 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:45,245 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:45,245 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:45,253 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001 doesn't exist and is created
2016-01-14 15:48:45,269 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:45,269 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:45,270 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001
2016-01-14 15:48:45,273 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786525245_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:45,282 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:45,283 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:45,285 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:45,285 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:45,285 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001/localmode-log-dir
2016-01-14 15:48:45,286 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001/localmode-log-dir/history.txt.appattempt_1452786525245_0001_000000, maxErrors=10
2016-01-14 15:48:45,286 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:45,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786525245_0001_000000, appSubmitTime=1452786525273, launchTime=1452786525273
2016-01-14 15:48:45,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:45,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:45,292 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:45,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:45,293 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41632
2016-01-14 15:48:45,298 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:45,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786525245_0001_000000, startTime=1452786525300
2016-01-14 15:48:45,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:45,300 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:45,369 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:45,370 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:45,370 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_true_false_false, applicationId=application_1452786525245_0001, dagName=[3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-01-14 15:48:45,385 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-01-14 15:48:45,386 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786525245_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001/localmode-log-dir/dag_1452786525245_0001_1.dot
2016-01-14 15:48:45,388 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001/localmode-log-dir/dag_1452786525245_0001_1-tez-dag.pb.txt
2016-01-14 15:48:45,396 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786525245_0001_1
2016-01-14 15:48:45,397 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:45,397 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-01-14 15:48:45 Running Dag: dag_1452786525245_0001_1
2016-01-14 15:48:45,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786525245_0001_1, submitTime=1452786525385
2016-01-14 15:48:45,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:45,412 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,412 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:45,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786525245_0001_1, initTime=1452786525411
2016-01-14 15:48:45,413 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786525245_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:45,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:45,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:DAG_STARTED]: dagID=dag_1452786525245_0001_1, startTime=1452786525414
2016-01-14 15:48:45,414 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:45,414 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786525245_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:45,414 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F91DF1D91E3D44FD8E3DE03FB6680831 : {4659393DEC1445798DE07708DCE17AC1={InputName=4659393DEC1445798DE07708DCE17AC1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:45,414 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4659393DEC1445798DE07708DCE17AC1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:45,415 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:45,415 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_true_false_false, applicationId=application_1452786525245_0001, dagName=[3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-01-14 15:48:45,415 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,417 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,417 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]. Starting root input initializers: 1
2016-01-14 15:48:45,417 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:45,417 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,418 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4659393DEC1445798DE07708DCE17AC1 on vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,420 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:45,420 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:45,420 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,420 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4659393DEC1445798DE07708DCE17AC1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:45,421 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,421 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,421 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9839DB7B36AA49A697CD6E8731DD6EA6, vertexId=vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:45,421 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:45,422 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:45,421 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9839DB7B36AA49A697CD6E8731DD6EA6, vertexId=vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,424 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9839DB7B36AA49A697CD6E8731DD6EA6, vertexName=CAA3E76E89A5451A89B58F6E4903A617, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:45,424 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9839DB7B36AA49A697CD6E8731DD6EA6, vertexId=vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,425 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:45,425 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4659393DEC1445798DE07708DCE17AC1 on vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:45,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAA3E76E89A5451A89B58F6E4903A617, vertexId=vertex_1452786525245_0001_1_01, initRequestedTime=1452786525417, initedTime=1452786525425, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:45,426 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:45,426 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:45,427 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4659393DEC1445798DE07708DCE17AC1=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:45,427 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] parallelism set to 1
2016-01-14 15:48:45,427 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:45,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786525245_0001_1_00, eventCount=1
2016-01-14 15:48:45,428 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:45,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, vertexId=vertex_1452786525245_0001_1_00, initRequestedTime=1452786525414, initedTime=1452786525428, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:45,429 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] with distanceFromRoot: 0
2016-01-14 15:48:45,429 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:45,429 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:45,429 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:45,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786525245_0001_1_00, startRequestedTime=1452786525429, startedTime=1452786525429
2016-01-14 15:48:45,430 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:45,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:45,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_STARTED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, taskId=task_1452786525245_0001_1_00_000000, scheduledTime=1452786525430, launchTime=1452786525430
2016-01-14 15:48:45,431 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:45,431 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786525245_0001_1_00 for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] numStartedSources: 1 numSources: 1
2016-01-14 15:48:45,431 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] with distanceFromRoot: 1
2016-01-14 15:48:45,431 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,432 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F91DF1D91E3D44FD8E3DE03FB6680831 in vertex: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,432 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CAA3E76E89A5451A89B58F6E4903A617 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:45,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:45,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786525245_0001_1_01, startRequestedTime=1452786525432, startedTime=1452786525432
2016-01-14 15:48:45,432 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:45,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexName: F91DF1D91E3D44FD8E3DE03FB6680831, VertexParallelism: 1, TaskAttemptID:attempt_1452786525245_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4659393DEC1445798DE07708DCE17AC1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAA3E76E89A5451A89B58F6E4903A617, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:45,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:45,433 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:45,434 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:45,434 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:45,434 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:45,434 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786525245_0001_00_000001
2016-01-14 15:48:45,435 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:45,436 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786525245_0001_00_000001
2016-01-14 15:48:45,436 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:45,436 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:45,436 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786525245_0001, with dag id: dag_1452786525245_0001_1
2016-01-14 15:48:45,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786525245_0001_00_000001, launchTime=1452786525436
2016-01-14 15:48:45,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:45,438 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786525245_0001_00_000001
2016-01-14 15:48:45,438 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786525245_0001_1_00_000000_0] to container: [container_1452786525245_0001_00_000001]
2016-01-14 15:48:45,439 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:45,439 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786525245_0001_1_00_000000_0] started. Is using containerId: [container_1452786525245_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:45,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0, startTime=1452786525439, containerId=container_1452786525245_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786525245_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:45,439 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:45,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:45,440 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:45,440 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786525245_0001_00_000001 given task: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,440 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786525245_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45 Starting to run new task attempt: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,441 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,441 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:45,442 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:45,442 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:45,442 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:45,442 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:45,442 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:45,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexName: F91DF1D91E3D44FD8E3DE03FB6680831, VertexParallelism: 1, TaskAttemptID:attempt_1452786525245_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4659393DEC1445798DE07708DCE17AC1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAA3E76E89A5451A89B58F6E4903A617, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:45,443 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:45,443 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:45,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:45,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4659393DEC1445798DE07708DCE17AC1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:45,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CAA3E76E89A5451A89B58F6E4903A617, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:45,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:45,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:45,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,450 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,449 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786525245_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:45,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:45,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,452 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:45,452 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:45,452 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,453 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:45,453 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,459 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F91DF1D91E3D44FD8E3DE03FB6680831, ordinal: 0
2016-01-14 15:48:45,459 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F91DF1D91E3D44FD8E3DE03FB6680831, mem on start (mb), free: 98, total: 447, max: 672
2016-01-14 15:48:45,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:45,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:45,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:45,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:45,460 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:45,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:45,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:45,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:45,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:45,461 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:45,461 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4659393DEC1445798DE07708DCE17AC1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:45,461 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CAA3E76E89A5451A89B58F6E4903A617, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:45,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:45,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4659393DEC1445798DE07708DCE17AC1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:45,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:45,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:45,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,462 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:45,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:45,462 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,463 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:45,463 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@638bab94
2016-01-14 15:48:45,464 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@638bab94
2016-01-14 15:48:45,464 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:45,464 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:45,466 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:45,467 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,467 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 299EE9166C0A49B3A97712D62670F619
2016-01-14 15:48:45,468 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 299EE9166C0A49B3A97712D62670F619
2016-01-14 15:48:45,469 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@12f61ade; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5b5dee8b; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@73d81689; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:45,470 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:45,470 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:45,470 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:45,492 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:45,492 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:45,492 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:45,492 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:45,493 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:45,493 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:45,493 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:45,494 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F91DF1D91E3D44FD8E3DE03FB6680831, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:45,498 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,498 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F91DF1D91E3D44FD8E3DE03FB6680831, mem on close (mb), free: 208, total: 451, max: 672
2016-01-14 15:48:45,498 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:45,498 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:45,499 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:45,499 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:46,643 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:46,643 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:47,001 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:47 Completed running task attempt: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:47,004 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:47,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786525245_0001_1_00, eventCount=1
2016-01-14 15:48:47,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786525494, Process_Duration=4, Process_End_Time=1452786525498, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:47,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:47,005 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:47,005 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,006 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786525245_0001_00_000001
2016-01-14 15:48:47,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0, startTime=1452786525439, finishTime=1452786527005, timeTaken=1566, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525494, Process_Duration=4, Process_End_Time=1452786525498, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:47,006 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:47,006 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:47,006 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:47,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_FINISHED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, taskId=task_1452786525245_0001_1_00_000000, startTime=1452786525439, finishTime=1452786527007, timeTaken=1568, status=SUCCEEDED, successfulAttemptID=attempt_1452786525245_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525494, Process_Duration=4, Process_End_Time=1452786525498, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:47,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:47,007 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:47,008 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] : 1
2016-01-14 15:48:47,008 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:47,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:47,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_FINISHED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, vertexId=vertex_1452786525245_0001_1_00, initRequestedTime=1452786525414, initedTime=1452786525428, startRequestedTime=1452786525429, startedTime=1452786525429, finishTime=1452786527008, timeTaken=1579, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525494, Process_Duration=4, Process_End_Time=1452786525498, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786525439, firstTasksToStart=[ task_1452786525245_0001_1_00_000000 ], lastTaskFinishTime=1452786527005, lastTasksToFinish=[ task_1452786525245_0001_1_00_000000 ], minTaskDuration=1566, maxTaskDuration=1566, avgTaskDuration=1566.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786525245_0001_1_00_000000 ], longestDurationTasks=[ task_1452786525245_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:47,009 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:47,009 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:47,009 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] attempt: attempt_1452786525245_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:47,010 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:47,010 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:47,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786525245_0001_00_000001, stoppedTime=1452786527010, exitStatus=0
2016-01-14 15:48:47,010 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:47,010 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:47,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_STARTED]: vertexName=CAA3E76E89A5451A89B58F6E4903A617, taskId=task_1452786525245_0001_1_01_000000, scheduledTime=1452786527010, launchTime=1452786527010
2016-01-14 15:48:47,011 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:47,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:47,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:47,011 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786525245_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:47,011 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:47,011 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:47,012 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexName: CAA3E76E89A5451A89B58F6E4903A617, VertexParallelism: 1, TaskAttemptID:attempt_1452786525245_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F91DF1D91E3D44FD8E3DE03FB6680831, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9839DB7B36AA49A697CD6E8731DD6EA6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:47,012 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:47,012 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:47,012 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786525245_0001_00_000002
2016-01-14 15:48:47,013 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:47,014 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786525245_0001_00_000002
2016-01-14 15:48:47,014 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:47,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:47,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786525245_0001_00_000002, launchTime=1452786527015
2016-01-14 15:48:47,016 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786525245_0001_00_000002
2016-01-14 15:48:47,016 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786525245_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:47,016 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786525245_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:47,017 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786525245_0001_00_000001, Exiting
2016-01-14 15:48:47,017 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786525245_0001_00_000001
2016-01-14 15:48:47,017 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786525245_0001_00_000001
2016-01-14 15:48:47,018 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:47,019 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786525245_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:47,020 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786525245_0001_00_000001 completed successfully
2016-01-14 15:48:47,021 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:47,027 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786525245_0001_1_01_000000_0] to container: [container_1452786525245_0001_00_000002]
2016-01-14 15:48:47,027 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:47,027 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786525245_0001_00_000002 given task: attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,028 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786525245_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,028 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786525245_0001_1_01_000000_0] started. Is using containerId: [container_1452786525245_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:47,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:47,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAA3E76E89A5451A89B58F6E4903A617, taskAttemptId=attempt_1452786525245_0001_1_01_000000_0, startTime=1452786527027, containerId=container_1452786525245_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786525245_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:47,029 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:47,030 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:47 Starting to run new task attempt: attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,031 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,031 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,031 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:47,031 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:47,031 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:47,031 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:47,031 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:47,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexName: CAA3E76E89A5451A89B58F6E4903A617, VertexParallelism: 1, TaskAttemptID:attempt_1452786525245_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F91DF1D91E3D44FD8E3DE03FB6680831, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9839DB7B36AA49A697CD6E8731DD6EA6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:47,032 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:47,032 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:47,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F91DF1D91E3D44FD8E3DE03FB6680831, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:47,035 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786525245_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:47,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:47,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:47,037 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:47,037 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:47,038 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:47,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:47,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,041 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:47,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9839DB7B36AA49A697CD6E8731DD6EA6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:47,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:47,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9839DB7B36AA49A697CD6E8731DD6EA6
2016-01-14 15:48:47,046 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:47,047 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9839DB7B36AA49A697CD6E8731DD6EA6
2016-01-14 15:48:47,052 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAA3E76E89A5451A89B58F6E4903A617, ordinal: 1
2016-01-14 15:48:47,052 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAA3E76E89A5451A89B58F6E4903A617, mem on start (mb), free: 145, total: 458, max: 672
2016-01-14 15:48:47,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:47,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:47,061 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:47,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9839DB7B36AA49A697CD6E8731DD6EA6
2016-01-14 15:48:47,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:47,062 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:47,062 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:47,062 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:47,062 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:47,062 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:47,062 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:47,063 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F91DF1D91E3D44FD8E3DE03FB6680831, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:47,063 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9839DB7B36AA49A697CD6E8731DD6EA6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:47,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:47,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:47,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F91DF1D91E3D44FD8E3DE03FB6680831 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:47,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:47,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:47,064 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:47,065 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:47,067 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F91DF1D91E3D44FD8E3DE03FB6680831 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:47,067 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:47,067 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:47,067 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:47,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:47,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:47,075 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,076 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786525245_0001_1_00_000000_0_10065, runDuration: 1551993, ]
2016-01-14 15:48:47,079 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:47,080 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 299EE9166C0A49B3A97712D62670F619
2016-01-14 15:48:47,080 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false"], id: 9839DB7B36AA49A697CD6E8731DD6EA6
2016-01-14 15:48:47,080 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false"]
2016-01-14 15:48:47,081 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 299EE9166C0A49B3A97712D62670F619, for 1 inputs
2016-01-14 15:48:47,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786525245_0001_1_00_000000_0_10065] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786527081, TimeTaken=2, Rate=0.35 MB/s
2016-01-14 15:48:47,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:47,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:47,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F91DF1D91E3D44FD8E3DE03FB6680831] #12 in 4ms
2016-01-14 15:48:47,083 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F91DF1D91E3D44FD8E3DE03FB6680831] #12, status:false, isInterrupted:false
2016-01-14 15:48:47,084 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:47,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:47,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:47,085 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:47,085 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:47,085 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:47,085 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:47,086 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAA3E76E89A5451A89B58F6E4903A617, all 1 inputs ready in: 00:00:00.005
2016-01-14 15:48:47,087 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@1f937be0; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@ba9bf52
2016-01-14 15:48:47,092 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:47,092 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:47,092 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,092 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786525245_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:47,093 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,093 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAA3E76E89A5451A89B58F6E4903A617, mem on close (mb), free: 141, total: 458, max: 672
2016-01-14 15:48:47,094 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:47,094 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:47,095 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:47,095 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:47,095 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:47,095 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:47 Completed running task attempt: attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,096 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786525245_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:47,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786527086, Process_Duration=7, Process_End_Time=1452786527093, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:47,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:47,098 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:47,098 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,098 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786525245_0001_00_000002
2016-01-14 15:48:47,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAA3E76E89A5451A89B58F6E4903A617, taskAttemptId=attempt_1452786525245_0001_1_01_000000_0, startTime=1452786527027, finishTime=1452786527097, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527086, Process_Duration=7, Process_End_Time=1452786527093, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:47,098 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:47,099 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:47,099 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786525245_0001_1_01_000000_0
2016-01-14 15:48:47,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_FINISHED]: vertexName=CAA3E76E89A5451A89B58F6E4903A617, taskId=task_1452786525245_0001_1_01_000000, startTime=1452786527027, finishTime=1452786527099, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1452786525245_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527086, Process_Duration=7, Process_End_Time=1452786527093, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:47,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:47,100 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:47,100 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] : 1
2016-01-14 15:48:47,101 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:47,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:47,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_FINISHED]: vertexName=CAA3E76E89A5451A89B58F6E4903A617, vertexId=vertex_1452786525245_0001_1_01, initRequestedTime=1452786525417, initedTime=1452786525425, startRequestedTime=1452786525432, startedTime=1452786525432, finishTime=1452786527101, timeTaken=1669, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527086, Process_Duration=7, Process_End_Time=1452786527093, Read_Duration=4, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786527027, firstTasksToStart=[ task_1452786525245_0001_1_01_000000 ], lastTaskFinishTime=1452786527097, lastTasksToFinish=[ task_1452786525245_0001_1_01_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786525245_0001_1_01_000000 ], longestDurationTasks=[ task_1452786525245_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:47,102 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:47,102 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:47,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786525245_0001_00_000002, stoppedTime=1452786527102, exitStatus=0
2016-01-14 15:48:47,102 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:47,102 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:47,103 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:47,103 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786525245_0001_1
2016-01-14 15:48:47,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786525245_0001_1
2016-01-14 15:48:47,104 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:47,104 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9839DB7B36AA49A697CD6E8731DD6EA6 for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:47,108 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786525245_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:47,109 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786525245_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:47,109 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786525245_0001_00_000002, Exiting
2016-01-14 15:48:47,109 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786525245_0001_00_000002
2016-01-14 15:48:47,109 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786525245_0001_00_000002
2016-01-14 15:48:47,109 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786525245_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:47,109 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786525245_0001_00_000002 completed successfully
2016-01-14 15:48:47,115 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786525245_0001_1, queueSize=0
2016-01-14 15:48:47,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:47,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786525245_0001_1, startTime=1452786525414, finishTime=1452786527113, timeTaken=1699, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=960495616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=20, FIRST_EVENT_RECEIVED=12, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573052580, Process_Duration=11, Process_End_Time=2905573052591, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:47,116 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786525245_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:47,116 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786525245_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:47,116 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:47,116 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786525245_0001_00_000002
2016-01-14 15:48:47 Completed Dag: dag_1452786525245_0001_1
2016-01-14 15:48:47,117 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786525245_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:47,117 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786525245_0001_1_post
2016-01-14 15:48:47,118 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,118 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:47,121 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_true_false_false, applicationId=application_1452786525245_0001
2016-01-14 15:48:47,123 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:47,123 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:47,123 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:47,123 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:47,123 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:47,125 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:47,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:47,125 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:47,125 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:47,125 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:47,126 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:47,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:47,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:47,131 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001
2016-01-14 15:48:47,131 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:01.904
2016-01-14 15:48:47,204 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:47,205 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:47,251 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 15:48:47,260 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-01-14 15:48:47,261 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:47,261 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:47,261 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:47,273 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:47,280 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false
2016-01-14 15:48:47,324 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:47,324 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:47,324 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false"]
2016-01-14 15:48:47,325 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:47,325 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:47,325 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:47,326 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ve/false_false_true_false
2016-01-14 15:48:47,326 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:47,335 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:47,339 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:47,339 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:47,339 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:47,348 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE/.tez/application_1452786527339_0001 doesn't exist and is created
2016-01-14 15:48:47,363 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:47,363 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:47,364 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE/.tez/application_1452786527339_0001
2016-01-14 15:48:47,367 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786527339_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:47,375 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:47,376 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:47,377 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:47,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:47,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE/.tez/application_1452786527339_0001/localmode-log-dir
2016-01-14 15:48:47,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE/.tez/application_1452786527339_0001/localmode-log-dir/history.txt.appattempt_1452786527339_0001_000000, maxErrors=10
2016-01-14 15:48:47,379 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:47,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786527339_0001_000000, appSubmitTime=1452786527367, launchTime=1452786527367
2016-01-14 15:48:47,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:47,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:47,388 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:47,389 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:47,389 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:34263
2016-01-14 15:48:47,391 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:47,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786527339_0001_000000, startTime=1452786527394
2016-01-14 15:48:47,394 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:47,394 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:47,463 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:47,464 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:47,464 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_false_true_false, applicationId=application_1452786527339_0001, dagName=[A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE] comprehensivesort/(1/1) ...ve/false_false_true_false
2016-01-14 15:48:47,476 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE] comprehensivesort/(1/1) ...ve/false_false_true_false
2016-01-14 15:48:47,477 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786527339_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE/.tez/application_1452786527339_0001/localmode-log-dir/dag_1452786527339_0001_1.dot
2016-01-14 15:48:47,478 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE/.tez/application_1452786527339_0001/localmode-log-dir/dag_1452786527339_0001_1-tez-dag.pb.txt
2016-01-14 15:48:47,485 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786527339_0001_1
2016-01-14 15:48:47,485 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,486 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE] comprehensivesort/(1/1) ...ve/false_false_true_false
2016-01-14 15:48:47 Running Dag: dag_1452786527339_0001_1
2016-01-14 15:48:47,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786527339_0001_1, submitTime=1452786527476
2016-01-14 15:48:47,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:47,498 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E6ED25BE50064160ABE401B7C5325BBB
2016-01-14 15:48:47,498 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:47,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:47,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786527339_0001_1, initTime=1452786527497
2016-01-14 15:48:47,499 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786527339_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:47,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:DAG_STARTED]: dagID=dag_1452786527339_0001_1, startTime=1452786527499
2016-01-14 15:48:47,499 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:47,499 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786527339_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:47,499 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F57D0795742C413F95D5503705B0C11B : {D6A86DDA175C4074A2D661E6C3D345BC={InputName=D6A86DDA175C4074A2D661E6C3D345BC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:47,500 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D6A86DDA175C4074A2D661E6C3D345BC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:47,500 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:47,500 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,500 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,501 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]. Starting root input initializers: 1
2016-01-14 15:48:47,501 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:47,501 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,501 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_false_true_false, applicationId=application_1452786527339_0001, dagName=[A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE] comprehensivesort/(1/1) ...ve/false_false_true_false
2016-01-14 15:48:47,503 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D6A86DDA175C4074A2D661E6C3D345BC on vertex vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,504 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:47,505 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,505 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,505 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,505 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B7F7289063D34679B65805C0D7EF1FF4, vertexId=vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:47,506 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:47,506 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B7F7289063D34679B65805C0D7EF1FF4, vertexId=vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,507 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D6A86DDA175C4074A2D661E6C3D345BC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:47,508 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:47,508 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:47,509 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B7F7289063D34679B65805C0D7EF1FF4, vertexName=E6ED25BE50064160ABE401B7C5325BBB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:47,509 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B7F7289063D34679B65805C0D7EF1FF4, vertexId=vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:47,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E6ED25BE50064160ABE401B7C5325BBB, vertexId=vertex_1452786527339_0001_1_01, initRequestedTime=1452786527501, initedTime=1452786527511, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:47,511 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:47,511 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:47,512 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D6A86DDA175C4074A2D661E6C3D345BC on vertex vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,512 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:47,512 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D6A86DDA175C4074A2D661E6C3D345BC=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:47,512 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] parallelism set to 1
2016-01-14 15:48:47,512 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:47,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786527339_0001_1_00, eventCount=1
2016-01-14 15:48:47,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:47,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F57D0795742C413F95D5503705B0C11B, vertexId=vertex_1452786527339_0001_1_00, initRequestedTime=1452786527499, initedTime=1452786527513, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:47,513 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] with distanceFromRoot: 0
2016-01-14 15:48:47,514 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:47,514 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,514 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,514 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:47,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786527339_0001_1_00, startRequestedTime=1452786527514, startedTime=1452786527514
2016-01-14 15:48:47,515 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786527339_0001, with dag id: dag_1452786527339_0001_1
2016-01-14 15:48:47,515 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:47,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:47,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:TASK_STARTED]: vertexName=F57D0795742C413F95D5503705B0C11B, taskId=task_1452786527339_0001_1_00_000000, scheduledTime=1452786527515, launchTime=1452786527515
2016-01-14 15:48:47,516 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527339_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:47,516 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786527339_0001_1_00 for vertex: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB] numStartedSources: 1 numSources: 1
2016-01-14 15:48:47,516 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB] with distanceFromRoot: 1
2016-01-14 15:48:47,516 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,517 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F57D0795742C413F95D5503705B0C11B in vertex: E6ED25BE50064160ABE401B7C5325BBB
2016-01-14 15:48:47,517 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E6ED25BE50064160ABE401B7C5325BBB with 1 source tasks and 1 pending tasks
2016-01-14 15:48:47,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786527339_0001_1_01, startRequestedTime=1452786527517, startedTime=1452786527517
2016-01-14 15:48:47,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:47,518 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:47,518 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE] comprehensivesort/(1/1) ...ve/false_false_true_false, VertexName: F57D0795742C413F95D5503705B0C11B, VertexParallelism: 1, TaskAttemptID:attempt_1452786527339_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D6A86DDA175C4074A2D661E6C3D345BC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E6ED25BE50064160ABE401B7C5325BBB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:47,518 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527339_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:47,518 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:47,519 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:47,519 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:47,519 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:47,520 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786527339_0001_00_000001
2016-01-14 15:48:47,520 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:47,522 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786527339_0001_00_000001
2016-01-14 15:48:47,522 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:47,523 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:47,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:47,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786527339_0001_00_000001, launchTime=1452786527523
2016-01-14 15:48:47,524 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786527339_0001_00_000001
2016-01-14 15:48:47,525 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786527339_0001_1_00_000000_0] to container: [container_1452786527339_0001_00_000001]
2016-01-14 15:48:47,525 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:47,525 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786527339_0001_1_00_000000_0] started. Is using containerId: [container_1452786527339_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:47,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F57D0795742C413F95D5503705B0C11B, taskAttemptId=attempt_1452786527339_0001_1_00_000000_0, startTime=1452786527525, containerId=container_1452786527339_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786527339_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:47,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:47,525 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527339_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:47,526 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527339_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:47,526 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786527339_0001_00_000001 given task: attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47,526 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786527339_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47 Starting to run new task attempt: attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47,526 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47,527 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,527 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:47,527 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:47,527 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:47,527 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:47,527 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:47,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE] comprehensivesort/(1/1) ...ve/false_false_true_false, VertexName: F57D0795742C413F95D5503705B0C11B, VertexParallelism: 1, TaskAttemptID:attempt_1452786527339_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D6A86DDA175C4074A2D661E6C3D345BC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E6ED25BE50064160ABE401B7C5325BBB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:47,527 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:47,528 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:47,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,528 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786527339_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:47,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E6ED25BE50064160ABE401B7C5325BBB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:47,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:47,531 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:47,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D6A86DDA175C4074A2D661E6C3D345BC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:47,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:47,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D6A86DDA175C4074A2D661E6C3D345BC
2016-01-14 15:48:47,532 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D6A86DDA175C4074A2D661E6C3D345BC
2016-01-14 15:48:47,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E6ED25BE50064160ABE401B7C5325BBB
2016-01-14 15:48:47,534 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:47,534 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D6A86DDA175C4074A2D661E6C3D345BC
2016-01-14 15:48:47,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D6A86DDA175C4074A2D661E6C3D345BC
2016-01-14 15:48:47,534 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:47,536 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E6ED25BE50064160ABE401B7C5325BBB
2016-01-14 15:48:47,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E6ED25BE50064160ABE401B7C5325BBB
2016-01-14 15:48:47,541 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F57D0795742C413F95D5503705B0C11B, ordinal: 0
2016-01-14 15:48:47,541 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F57D0795742C413F95D5503705B0C11B, mem on start (mb), free: 269, total: 458, max: 672
2016-01-14 15:48:47,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:47,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:47,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:47,542 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:47,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:47,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:47,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:47,543 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:47,544 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:47,544 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D6A86DDA175C4074A2D661E6C3D345BC, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:47,544 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E6ED25BE50064160ABE401B7C5325BBB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:47,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:47,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D6A86DDA175C4074A2D661E6C3D345BC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:47,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:47,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:47,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D6A86DDA175C4074A2D661E6C3D345BC
2016-01-14 15:48:47,546 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:47,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D6A86DDA175C4074A2D661E6C3D345BC
2016-01-14 15:48:47,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:47,547 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47,547 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:47,548 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1e55a4ef
2016-01-14 15:48:47,549 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1e55a4ef
2016-01-14 15:48:47,549 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:47,550 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:47,549 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:47,551 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: D6A86DDA175C4074A2D661E6C3D345BC
2016-01-14 15:48:47,551 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 632E87602DB141C7AEA02FC9F9566EF4
2016-01-14 15:48:47,551 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 632E87602DB141C7AEA02FC9F9566EF4
2016-01-14 15:48:47,551 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@172a3828; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@54ccd5f9; comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@53f3f9ed; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:47,552 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:47,552 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:47,552 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:47,566 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:47,566 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:47,566 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:47,566 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:47,567 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:47,567 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:47,567 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:47,568 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F57D0795742C413F95D5503705B0C11B, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:47,573 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47,573 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F57D0795742C413F95D5503705B0C11B, mem on close (mb), free: 167, total: 458, max: 672
2016-01-14 15:48:47,574 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:47,574 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:47,574 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:47,574 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:47,577 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:47 Completed running task attempt: attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47,580 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786527339_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:47,581 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786527339_0001_1_00, eventCount=1
2016-01-14 15:48:47,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:47,581 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F57D0795742C413F95D5503705B0C11B, taskAttemptId=attempt_1452786527339_0001_1_00_000000_0, startTime=1452786527525, finishTime=1452786527581, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527568, Process_Duration=5, Process_End_Time=1452786527573, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:47,581 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527339_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:47,581 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786527339_0001_1_00_000000_0
2016-01-14 15:48:47,582 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:47,582 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:47,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:TASK_FINISHED]: vertexName=F57D0795742C413F95D5503705B0C11B, taskId=task_1452786527339_0001_1_00_000000, startTime=1452786527525, finishTime=1452786527582, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1452786527339_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527568, Process_Duration=5, Process_End_Time=1452786527573, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:47,583 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527339_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:47,583 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] : 1
2016-01-14 15:48:47,583 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,584 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:47,584 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:VERTEX_FINISHED]: vertexName=F57D0795742C413F95D5503705B0C11B, vertexId=vertex_1452786527339_0001_1_00, initRequestedTime=1452786527499, initedTime=1452786527513, startRequestedTime=1452786527514, startedTime=1452786527514, finishTime=1452786527583, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527568, Process_Duration=5, Process_End_Time=1452786527573, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786527525, firstTasksToStart=[ task_1452786527339_0001_1_00_000000 ], lastTaskFinishTime=1452786527581, lastTasksToFinish=[ task_1452786527339_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786527339_0001_1_00_000000 ], longestDurationTasks=[ task_1452786527339_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:47,584 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:47,585 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:47,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786527339_0001_00_000001, stoppedTime=1452786527585, exitStatus=0
2016-01-14 15:48:47,585 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:47,585 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB] attempt: attempt_1452786527339_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:47,585 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E6ED25BE50064160ABE401B7C5325BBB
2016-01-14 15:48:47,585 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,586 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:47,586 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786527339_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:47,585 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:47,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786527568, Process_Duration=5, Process_End_Time=1452786527573, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:47,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:47,586 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:47,587 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:TASK_STARTED]: vertexName=E6ED25BE50064160ABE401B7C5325BBB, taskId=task_1452786527339_0001_1_01_000000, scheduledTime=1452786527586, launchTime=1452786527586
2016-01-14 15:48:47,587 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527339_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:47,587 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:47,587 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,587 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786527339_0001_00_000001
2016-01-14 15:48:47,587 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE] comprehensivesort/(1/1) ...ve/false_false_true_false, VertexName: E6ED25BE50064160ABE401B7C5325BBB, VertexParallelism: 1, TaskAttemptID:attempt_1452786527339_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F57D0795742C413F95D5503705B0C11B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B7F7289063D34679B65805C0D7EF1FF4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:47,588 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527339_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:47,588 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:47,588 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786527339_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:47,588 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786527339_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:47,588 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786527339_0001_00_000001, Exiting
2016-01-14 15:48:47,588 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786527339_0001_00_000001
2016-01-14 15:48:47,588 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786527339_0001_00_000001
2016-01-14 15:48:47,590 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786527339_0001_00_000002
2016-01-14 15:48:47,590 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:47,590 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786527339_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:47,590 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786527339_0001_00_000001 completed successfully
2016-01-14 15:48:47,591 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786527339_0001_00_000002
2016-01-14 15:48:47,591 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:47,591 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:47,593 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:47,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:47,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786527339_0001_00_000002, launchTime=1452786527593
2016-01-14 15:48:47,594 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786527339_0001_00_000002
2016-01-14 15:48:47,595 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786527339_0001_1_01_000000_0] to container: [container_1452786527339_0001_00_000002]
2016-01-14 15:48:47,596 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:47,596 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786527339_0001_00_000002 given task: attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,597 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786527339_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,597 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786527339_0001_1_01_000000_0] started. Is using containerId: [container_1452786527339_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:47,597 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E6ED25BE50064160ABE401B7C5325BBB, taskAttemptId=attempt_1452786527339_0001_1_01_000000_0, startTime=1452786527596, containerId=container_1452786527339_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786527339_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:47,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:47 Starting to run new task attempt: attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,597 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527339_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:47,597 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527339_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:47,597 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,598 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,598 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:47,598 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:47,598 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:47,598 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:47,598 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:47,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE] comprehensivesort/(1/1) ...ve/false_false_true_false, VertexName: E6ED25BE50064160ABE401B7C5325BBB, VertexParallelism: 1, TaskAttemptID:attempt_1452786527339_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F57D0795742C413F95D5503705B0C11B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B7F7289063D34679B65805C0D7EF1FF4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:47,599 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:47,599 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:47,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,600 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786527339_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:47,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F57D0795742C413F95D5503705B0C11B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:47,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B7F7289063D34679B65805C0D7EF1FF4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:47,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:47,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:47,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B7F7289063D34679B65805C0D7EF1FF4
2016-01-14 15:48:47,602 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:47,602 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B7F7289063D34679B65805C0D7EF1FF4
2016-01-14 15:48:47,603 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:47,603 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:47,603 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,607 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:47,614 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:47,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B7F7289063D34679B65805C0D7EF1FF4
2016-01-14 15:48:47,617 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E6ED25BE50064160ABE401B7C5325BBB, ordinal: 1
2016-01-14 15:48:47,617 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E6ED25BE50064160ABE401B7C5325BBB, mem on start (mb), free: 160, total: 458, max: 672
2016-01-14 15:48:47,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:47,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:47,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:47,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:47,619 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:47,620 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:47,620 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:47,620 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:47,621 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:47,621 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:47,621 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B7F7289063D34679B65805C0D7EF1FF4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:47,622 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F57D0795742C413F95D5503705B0C11B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:47,622 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:47,622 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F57D0795742C413F95D5503705B0C11B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:47,622 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:47,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:47,623 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:47,624 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:47,624 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F57D0795742C413F95D5503705B0C11B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:47,624 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:47,624 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:47,625 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:47,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:47,627 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,628 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786527339_0001_1_00_000000_0_10070, runDuration: 46265, ]
2016-01-14 15:48:47,630 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786527339_0001_1_00_000000_0_10070] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786527630, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:47,630 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,630 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:47,630 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:47,631 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F57D0795742C413F95D5503705B0C11B] #13 in 2ms
2016-01-14 15:48:47,631 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F57D0795742C413F95D5503705B0C11B] #13, status:false, isInterrupted:false
2016-01-14 15:48:47,631 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 632E87602DB141C7AEA02FC9F9566EF4
2016-01-14 15:48:47,632 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:47,632 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false"], id: B7F7289063D34679B65805C0D7EF1FF4
2016-01-14 15:48:47,632 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:47,632 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false"]
2016-01-14 15:48:47,632 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:47,632 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 632E87602DB141C7AEA02FC9F9566EF4, for 1 inputs
2016-01-14 15:48:47,632 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:47,633 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:47,633 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,633 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:47,633 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E6ED25BE50064160ABE401B7C5325BBB, all 1 inputs ready in: 00:00:00.001
2016-01-14 15:48:47,634 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@73eba297; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@72ffa2f5
2016-01-14 15:48:47,640 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:47,640 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:47,641 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,641 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786527339_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:47,642 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,642 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E6ED25BE50064160ABE401B7C5325BBB, mem on close (mb), free: 156, total: 458, max: 672
2016-01-14 15:48:47,642 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F57D0795742C413F95D5503705B0C11B
2016-01-14 15:48:47,643 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:47,643 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:47,643 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:47,644 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:47,644 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:47 Completed running task attempt: attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,645 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786527339_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:47,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786527634, Process_Duration=8, Process_End_Time=1452786527642, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:47,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E6ED25BE50064160ABE401B7C5325BBB, taskAttemptId=attempt_1452786527339_0001_1_01_000000_0, startTime=1452786527596, finishTime=1452786527646, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527634, Process_Duration=8, Process_End_Time=1452786527642, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:47,647 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527339_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:47,647 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786527339_0001_1_01_000000_0
2016-01-14 15:48:47,647 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:47,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:47,648 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:47,648 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:47,648 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:TASK_FINISHED]: vertexName=E6ED25BE50064160ABE401B7C5325BBB, taskId=task_1452786527339_0001_1_01_000000, startTime=1452786527596, finishTime=1452786527647, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452786527339_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527634, Process_Duration=8, Process_End_Time=1452786527642, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:47,648 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,648 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527339_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:47,649 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB] : 1
2016-01-14 15:48:47,649 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,650 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:47,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:VERTEX_FINISHED]: vertexName=E6ED25BE50064160ABE401B7C5325BBB, vertexId=vertex_1452786527339_0001_1_01, initRequestedTime=1452786527501, initedTime=1452786527511, startRequestedTime=1452786527517, startedTime=1452786527517, finishTime=1452786527649, timeTaken=132, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786527634, Process_Duration=8, Process_End_Time=1452786527642, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786527596, firstTasksToStart=[ task_1452786527339_0001_1_01_000000 ], lastTaskFinishTime=1452786527646, lastTasksToFinish=[ task_1452786527339_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786527339_0001_1_01_000000 ], longestDurationTasks=[ task_1452786527339_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:47,650 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:47,650 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:47,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786527339_0001_00_000002, stoppedTime=1452786527650, exitStatus=0
2016-01-14 15:48:47,651 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:47,651 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:47,651 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786527339_0001_1
2016-01-14 15:48:47,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786527339_0001_1
2016-01-14 15:48:47,651 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B7F7289063D34679B65805C0D7EF1FF4 for vertex: vertex_1452786527339_0001_1_01 [E6ED25BE50064160ABE401B7C5325BBB]
2016-01-14 15:48:47,649 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786527339_0001_00_000002
2016-01-14 15:48:47,650 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:47,653 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786527339_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:47,653 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786527339_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:47,653 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786527339_0001_00_000002, Exiting
2016-01-14 15:48:47,653 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786527339_0001_00_000002
2016-01-14 15:48:47,653 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786527339_0001_00_000002
2016-01-14 15:48:47,654 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786527339_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:47,654 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786527339_0001_00_000002 completed successfully
2016-01-14 15:48:47,660 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786527339_0001_1_00 [F57D0795742C413F95D5503705B0C11B]
2016-01-14 15:48:47,662 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786527339_0001_1, queueSize=0
2016-01-14 15:48:47,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:47,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527339_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786527339_0001_1, startTime=1452786527499, finishTime=1452786527660, timeTaken=161, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=960495616, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573055202, Process_Duration=13, Process_End_Time=2905573055215, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:47,664 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786527339_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:47,664 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786527339_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:47,664 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786527339_0001_00_000002
2016-01-14 15:48:47,664 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527339_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:47 Completed Dag: dag_1452786527339_0001_1
2016-01-14 15:48:47,665 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786527339_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:47,665 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786527339_0001_1_post
2016-01-14 15:48:47,666 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,666 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:47,668 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_false_true_false, applicationId=application_1452786527339_0001
2016-01-14 15:48:47,670 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:47,670 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:47,670 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:47,670 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:47,671 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:47,671 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:47,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:47,672 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:47,672 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:47,672 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:47,672 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:47,672 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:47,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:47,679 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A589FF4A146C4A088DE2F22069651602/C9F2C53D858B49CD9C60B5E83F9482EE/.tez/application_1452786527339_0001
2016-01-14 15:48:47,680 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.355
2016-01-14 15:48:47,733 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:47,734 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:47,773 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:48:47,777 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 15:48:47,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:47,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:47,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:47,790 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:47,797 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false
2016-01-14 15:48:47,833 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:47,833 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:47,834 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false"]
2016-01-14 15:48:47,835 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:47,835 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:47,835 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:47,835 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/true_false_true_false
2016-01-14 15:48:47,836 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:47,844 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:47,848 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:47,849 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:47,849 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:47,857 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E/.tez/application_1452786527849_0001 doesn't exist and is created
2016-01-14 15:48:47,872 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:47,873 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:47,873 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E/.tez/application_1452786527849_0001
2016-01-14 15:48:47,881 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786527849_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:47,889 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:47,889 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:47,892 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:47,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:47,893 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E/.tez/application_1452786527849_0001/localmode-log-dir
2016-01-14 15:48:47,893 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E/.tez/application_1452786527849_0001/localmode-log-dir/history.txt.appattempt_1452786527849_0001_000000, maxErrors=10
2016-01-14 15:48:47,893 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:47,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786527849_0001_000000, appSubmitTime=1452786527881, launchTime=1452786527881
2016-01-14 15:48:47,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:47,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:47,906 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:47,908 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:47,909 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44308
2016-01-14 15:48:47,912 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:47,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786527849_0001_000000, startTime=1452786527914
2016-01-14 15:48:47,914 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:47,914 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:47,973 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:47,973 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:47,973 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_false_true_false, applicationId=application_1452786527849_0001, dagName=[36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E] comprehensivesort/(1/1) ...ive/true_false_true_false
2016-01-14 15:48:47,985 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E] comprehensivesort/(1/1) ...ive/true_false_true_false
2016-01-14 15:48:47,987 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786527849_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E/.tez/application_1452786527849_0001/localmode-log-dir/dag_1452786527849_0001_1.dot
2016-01-14 15:48:47,987 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E/.tez/application_1452786527849_0001/localmode-log-dir/dag_1452786527849_0001_1-tez-dag.pb.txt
2016-01-14 15:48:47,995 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786527849_0001_1
2016-01-14 15:48:47,995 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,996 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E] comprehensivesort/(1/1) ...ive/true_false_true_false
2016-01-14 15:48:47 Running Dag: dag_1452786527849_0001_1
2016-01-14 15:48:48,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:48,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786527849_0001_1, submitTime=1452786527986
2016-01-14 15:48:48,009 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DACF065D8A0D45B081A85ED51CC147CD
2016-01-14 15:48:48,009 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:48,010 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:48,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786527849_0001_1, initTime=1452786528009
2016-01-14 15:48:48,010 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786527849_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:48,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:DAG_STARTED]: dagID=dag_1452786527849_0001_1, startTime=1452786528010
2016-01-14 15:48:48,011 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:48,011 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786527849_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:48,011 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D868CBC29DA640CBB3237CBE9102B94D : {23A7A2D9B8B64D459E49BA5171BE42F4={InputName=23A7A2D9B8B64D459E49BA5171BE42F4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:48,011 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 23A7A2D9B8B64D459E49BA5171BE42F4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:48,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:48,011 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:48,012 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:48,012 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:48,012 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_false_true_false, applicationId=application_1452786527849_0001, dagName=[36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E] comprehensivesort/(1/1) ...ive/true_false_true_false
2016-01-14 15:48:48,012 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]. Starting root input initializers: 1
2016-01-14 15:48:48,013 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:48,013 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:48,016 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:48,016 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:48,016 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 23A7A2D9B8B64D459E49BA5171BE42F4 on vertex vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:48,016 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:48,018 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:48,018 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3F8A50AC3C3B43C7AB05F2AB25B567B4, vertexId=vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:48,018 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3F8A50AC3C3B43C7AB05F2AB25B567B4, vertexId=vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:48,020 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3F8A50AC3C3B43C7AB05F2AB25B567B4, vertexName=DACF065D8A0D45B081A85ED51CC147CD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:48,021 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3F8A50AC3C3B43C7AB05F2AB25B567B4, vertexId=vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:48,021 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:48,021 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 23A7A2D9B8B64D459E49BA5171BE42F4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:48,022 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:48,022 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:48,023 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:48,023 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 23A7A2D9B8B64D459E49BA5171BE42F4 on vertex vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:48,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:48,027 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786527849_0001, with dag id: dag_1452786527849_0001_1
2016-01-14 15:48:48,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DACF065D8A0D45B081A85ED51CC147CD, vertexId=vertex_1452786527849_0001_1_01, initRequestedTime=1452786528013, initedTime=1452786528025, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:48,027 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:48,027 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:48,028 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {23A7A2D9B8B64D459E49BA5171BE42F4=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:48,029 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] parallelism set to 1
2016-01-14 15:48:48,029 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:48,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786527849_0001_1_00, eventCount=1
2016-01-14 15:48:48,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D868CBC29DA640CBB3237CBE9102B94D, vertexId=vertex_1452786527849_0001_1_00, initRequestedTime=1452786528011, initedTime=1452786528030, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:48,030 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] with distanceFromRoot: 0
2016-01-14 15:48:48,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:48,030 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:48,030 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:48,031 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:48,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786527849_0001_1_00, startRequestedTime=1452786528030, startedTime=1452786528030
2016-01-14 15:48:48,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:48,031 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:48,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:48,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:TASK_STARTED]: vertexName=D868CBC29DA640CBB3237CBE9102B94D, taskId=task_1452786527849_0001_1_00_000000, scheduledTime=1452786528034, launchTime=1452786528034
2016-01-14 15:48:48,034 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527849_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:48,034 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786527849_0001_1_00 for vertex: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD] numStartedSources: 1 numSources: 1
2016-01-14 15:48:48,035 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD] with distanceFromRoot: 1
2016-01-14 15:48:48,035 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:48,036 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D868CBC29DA640CBB3237CBE9102B94D in vertex: DACF065D8A0D45B081A85ED51CC147CD
2016-01-14 15:48:48,036 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DACF065D8A0D45B081A85ED51CC147CD with 1 source tasks and 1 pending tasks
2016-01-14 15:48:48,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786527849_0001_1_01, startRequestedTime=1452786528036, startedTime=1452786528036
2016-01-14 15:48:48,037 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:48,037 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:48,038 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E] comprehensivesort/(1/1) ...ive/true_false_true_false, VertexName: D868CBC29DA640CBB3237CBE9102B94D, VertexParallelism: 1, TaskAttemptID:attempt_1452786527849_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=23A7A2D9B8B64D459E49BA5171BE42F4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DACF065D8A0D45B081A85ED51CC147CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:48,038 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527849_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:48,038 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:48,039 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:48,040 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:48,040 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:48,040 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786527849_0001_00_000001
2016-01-14 15:48:48,040 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:48,042 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786527849_0001_00_000001
2016-01-14 15:48:48,042 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:48,043 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:48,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:48,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786527849_0001_00_000001, launchTime=1452786528043
2016-01-14 15:48:48,045 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786527849_0001_00_000001
2016-01-14 15:48:48,045 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786527849_0001_1_00_000000_0] to container: [container_1452786527849_0001_00_000001]
2016-01-14 15:48:48,045 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:48,045 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786527849_0001_00_000001 given task: attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:48,046 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786527849_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:48,046 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786527849_0001_1_00_000000_0] started. Is using containerId: [container_1452786527849_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:48,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:48,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D868CBC29DA640CBB3237CBE9102B94D, taskAttemptId=attempt_1452786527849_0001_1_00_000000_0, startTime=1452786528046, containerId=container_1452786527849_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786527849_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:48,047 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527849_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:48,047 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527849_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:48 Starting to run new task attempt: attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:48,048 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:48,048 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:48,048 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:48,048 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:48,049 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:48,049 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:48,049 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:48,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E] comprehensivesort/(1/1) ...ive/true_false_true_false, VertexName: D868CBC29DA640CBB3237CBE9102B94D, VertexParallelism: 1, TaskAttemptID:attempt_1452786527849_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=23A7A2D9B8B64D459E49BA5171BE42F4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DACF065D8A0D45B081A85ED51CC147CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:48,050 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:48,050 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:48,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:48,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:48,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=23A7A2D9B8B64D459E49BA5171BE42F4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:48,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:48,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 23A7A2D9B8B64D459E49BA5171BE42F4
2016-01-14 15:48:48,053 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 23A7A2D9B8B64D459E49BA5171BE42F4
2016-01-14 15:48:48,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:48,055 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786527849_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:48,055 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:48,057 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 23A7A2D9B8B64D459E49BA5171BE42F4
2016-01-14 15:48:48,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 23A7A2D9B8B64D459E49BA5171BE42F4
2016-01-14 15:48:48,057 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:48,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DACF065D8A0D45B081A85ED51CC147CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:48,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:48,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DACF065D8A0D45B081A85ED51CC147CD
2016-01-14 15:48:48,061 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:48,062 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DACF065D8A0D45B081A85ED51CC147CD
2016-01-14 15:48:48,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DACF065D8A0D45B081A85ED51CC147CD
2016-01-14 15:48:48,065 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D868CBC29DA640CBB3237CBE9102B94D, ordinal: 0
2016-01-14 15:48:48,065 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D868CBC29DA640CBB3237CBE9102B94D, mem on start (mb), free: 292, total: 458, max: 672
2016-01-14 15:48:48,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:48,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:48,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:48,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:48,066 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:48,066 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:48,066 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:48,066 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:48,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:48,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:48,067 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 23A7A2D9B8B64D459E49BA5171BE42F4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:48,067 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DACF065D8A0D45B081A85ED51CC147CD, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:48,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:48,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 23A7A2D9B8B64D459E49BA5171BE42F4
2016-01-14 15:48:48,068 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:48,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 23A7A2D9B8B64D459E49BA5171BE42F4
2016-01-14 15:48:48,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 23A7A2D9B8B64D459E49BA5171BE42F4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:48,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:48,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:48,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:48,069 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:48,069 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:48,070 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@270dcbd0
2016-01-14 15:48:48,072 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@270dcbd0
2016-01-14 15:48:48,075 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:48,075 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:48,075 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:48,075 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 23A7A2D9B8B64D459E49BA5171BE42F4
2016-01-14 15:48:48,076 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: F8A78E0640A740529FCD39271AD54587
2016-01-14 15:48:48,076 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] F8A78E0640A740529FCD39271AD54587
2016-01-14 15:48:48,077 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@206270ed; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1ee3e837; comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@4a282b9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:48,077 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:48,077 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:48,078 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:48,091 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:48,091 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:48,092 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:48,092 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:48,092 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:48,093 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:48,093 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:48,093 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D868CBC29DA640CBB3237CBE9102B94D, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:48,098 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:48,098 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D868CBC29DA640CBB3237CBE9102B94D, mem on close (mb), free: 190, total: 458, max: 672
2016-01-14 15:48:48,098 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:48,098 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:48,099 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:48,099 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:50,064 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:50,064 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:50,489 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:50 Completed running task attempt: attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:50,494 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786527849_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:50,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786527849_0001_1_00, eventCount=1
2016-01-14 15:48:50,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=339, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=485490688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786528093, Process_Duration=5, Process_End_Time=1452786528098, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:50,495 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:50,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D868CBC29DA640CBB3237CBE9102B94D, taskAttemptId=attempt_1452786527849_0001_1_00_000000_0, startTime=1452786528046, finishTime=1452786530495, timeTaken=2449, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=339, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=485490688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786528093, Process_Duration=5, Process_End_Time=1452786528098, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:50,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:50,495 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527849_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:50,495 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:50,495 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786527849_0001_1_00_000000_0
2016-01-14 15:48:50,495 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786527849_0001_00_000001
2016-01-14 15:48:50,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:50,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:TASK_FINISHED]: vertexName=D868CBC29DA640CBB3237CBE9102B94D, taskId=task_1452786527849_0001_1_00_000000, startTime=1452786528046, finishTime=1452786530495, timeTaken=2449, status=SUCCEEDED, successfulAttemptID=attempt_1452786527849_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=339, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=485490688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786528093, Process_Duration=5, Process_End_Time=1452786528098, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:50,496 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527849_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:50,496 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] : 1
2016-01-14 15:48:50,496 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:50,497 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:50,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:50,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:VERTEX_FINISHED]: vertexName=D868CBC29DA640CBB3237CBE9102B94D, vertexId=vertex_1452786527849_0001_1_00, initRequestedTime=1452786528011, initedTime=1452786528030, startRequestedTime=1452786528030, startedTime=1452786528030, finishTime=1452786530496, timeTaken=2466, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=339, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=485490688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786528093, Process_Duration=5, Process_End_Time=1452786528098, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786528046, firstTasksToStart=[ task_1452786527849_0001_1_00_000000 ], lastTaskFinishTime=1452786530495, lastTasksToFinish=[ task_1452786527849_0001_1_00_000000 ], minTaskDuration=2449, maxTaskDuration=2449, avgTaskDuration=2449.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786527849_0001_1_00_000000 ], longestDurationTasks=[ task_1452786527849_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:50,498 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:50,498 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD] attempt: attempt_1452786527849_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:50,499 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DACF065D8A0D45B081A85ED51CC147CD
2016-01-14 15:48:50,499 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:50,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:50,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786527849_0001_00_000001, stoppedTime=1452786530500, exitStatus=0
2016-01-14 15:48:50,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:50,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:50,501 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:50,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:50,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:TASK_STARTED]: vertexName=DACF065D8A0D45B081A85ED51CC147CD, taskId=task_1452786527849_0001_1_01_000000, scheduledTime=1452786530501, launchTime=1452786530501
2016-01-14 15:48:50,501 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527849_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:50,502 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786527849_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:50,502 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:50,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:50,502 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E] comprehensivesort/(1/1) ...ive/true_false_true_false, VertexName: DACF065D8A0D45B081A85ED51CC147CD, VertexParallelism: 1, TaskAttemptID:attempt_1452786527849_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D868CBC29DA640CBB3237CBE9102B94D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3F8A50AC3C3B43C7AB05F2AB25B567B4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:50,503 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527849_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:50,503 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:50,503 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786527849_0001_00_000002
2016-01-14 15:48:50,504 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:50,505 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786527849_0001_00_000002
2016-01-14 15:48:50,505 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:50,506 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786527849_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:50,506 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786527849_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:50,506 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786527849_0001_00_000001, Exiting
2016-01-14 15:48:50,506 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786527849_0001_00_000001
2016-01-14 15:48:50,506 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786527849_0001_00_000001
2016-01-14 15:48:50,508 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:50,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:50,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786527849_0001_00_000002, launchTime=1452786530508
2016-01-14 15:48:50,509 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786527849_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:50,509 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786527849_0001_00_000001 completed successfully
2016-01-14 15:48:50,510 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:50,509 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786527849_0001_00_000002
2016-01-14 15:48:50,510 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786527849_0001_1_01_000000_0] to container: [container_1452786527849_0001_00_000002]
2016-01-14 15:48:50,511 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:50,511 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786527849_0001_00_000002 given task: attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,511 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786527849_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,511 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786527849_0001_1_01_000000_0] started. Is using containerId: [container_1452786527849_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:50,511 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DACF065D8A0D45B081A85ED51CC147CD, taskAttemptId=attempt_1452786527849_0001_1_01_000000_0, startTime=1452786530511, containerId=container_1452786527849_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786527849_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:50 Starting to run new task attempt: attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:50,512 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,512 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:50,512 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:50,512 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:50,512 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:50,512 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:50,512 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:50,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E] comprehensivesort/(1/1) ...ive/true_false_true_false, VertexName: DACF065D8A0D45B081A85ED51CC147CD, VertexParallelism: 1, TaskAttemptID:attempt_1452786527849_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D868CBC29DA640CBB3237CBE9102B94D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3F8A50AC3C3B43C7AB05F2AB25B567B4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:50,513 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:50,512 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527849_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:50,515 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527849_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:50,516 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786527849_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:50,516 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:50,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:50,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:50,520 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:50,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3F8A50AC3C3B43C7AB05F2AB25B567B4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:50,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:50,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3F8A50AC3C3B43C7AB05F2AB25B567B4
2016-01-14 15:48:50,523 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:50,523 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3F8A50AC3C3B43C7AB05F2AB25B567B4
2016-01-14 15:48:50,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D868CBC29DA640CBB3237CBE9102B94D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:50,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:50,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:50,528 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:50,528 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:50,529 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:50,531 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DACF065D8A0D45B081A85ED51CC147CD, ordinal: 1
2016-01-14 15:48:50,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:50,531 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DACF065D8A0D45B081A85ED51CC147CD, mem on start (mb), free: 215, total: 463, max: 672
2016-01-14 15:48:50,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:50,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:50,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:50,539 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:50,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3F8A50AC3C3B43C7AB05F2AB25B567B4
2016-01-14 15:48:50,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:50,540 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:50,540 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:50,540 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:50,540 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:50,540 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:50,541 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:50,541 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3F8A50AC3C3B43C7AB05F2AB25B567B4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:50,541 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D868CBC29DA640CBB3237CBE9102B94D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:50,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:50,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D868CBC29DA640CBB3237CBE9102B94D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:50,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:50,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:50,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:50,543 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:50,543 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:50,544 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D868CBC29DA640CBB3237CBE9102B94D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:50,544 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:50,544 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:50,544 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:50,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:50,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:50,546 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,548 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786527849_0001_1_00_000000_0_10077, runDuration: 2431619, ]
2016-01-14 15:48:50,551 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786527849_0001_1_00_000000_0_10077] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786530551, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:50,551 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:50,551 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:50,551 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:50,552 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D868CBC29DA640CBB3237CBE9102B94D] #14 in 2ms
2016-01-14 15:48:50,552 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: F8A78E0640A740529FCD39271AD54587
2016-01-14 15:48:50,552 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false"], id: 3F8A50AC3C3B43C7AB05F2AB25B567B4
2016-01-14 15:48:50,553 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false"]
2016-01-14 15:48:50,553 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] F8A78E0640A740529FCD39271AD54587, for 1 inputs
2016-01-14 15:48:50,553 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D868CBC29DA640CBB3237CBE9102B94D] #14, status:false, isInterrupted:false
2016-01-14 15:48:50,554 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:50,555 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:50,555 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:50,555 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:50,556 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:50,556 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:50,556 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DACF065D8A0D45B081A85ED51CC147CD, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:48:50,556 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:50,557 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@35a499ce; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@496dbee5
2016-01-14 15:48:50,561 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:50,561 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:50,562 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,562 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786527849_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:50,562 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,562 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DACF065D8A0D45B081A85ED51CC147CD, mem on close (mb), free: 211, total: 463, max: 672
2016-01-14 15:48:50,563 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D868CBC29DA640CBB3237CBE9102B94D
2016-01-14 15:48:50,563 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:50,563 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:50,564 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:50,564 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:50,564 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:50 Completed running task attempt: attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,565 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786527849_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:50,566 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:50,566 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DACF065D8A0D45B081A85ED51CC147CD, taskAttemptId=attempt_1452786527849_0001_1_01_000000_0, startTime=1452786530511, finishTime=1452786530566, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=485490688, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786530556, Process_Duration=6, Process_End_Time=1452786530562, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:50,566 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786527849_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:50,567 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:50,567 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786527849_0001_1_01_000000_0
2016-01-14 15:48:50,567 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:50,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:TASK_FINISHED]: vertexName=DACF065D8A0D45B081A85ED51CC147CD, taskId=task_1452786527849_0001_1_01_000000, startTime=1452786530511, finishTime=1452786530567, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1452786527849_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=485490688, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786530556, Process_Duration=6, Process_End_Time=1452786530562, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:50,568 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786527849_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:50,568 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD] : 1
2016-01-14 15:48:50,568 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:50,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=485490688, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786530556, Process_Duration=6, Process_End_Time=1452786530562, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:50,569 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:50,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:50,569 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:50,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:VERTEX_FINISHED]: vertexName=DACF065D8A0D45B081A85ED51CC147CD, vertexId=vertex_1452786527849_0001_1_01, initRequestedTime=1452786528013, initedTime=1452786528025, startRequestedTime=1452786528036, startedTime=1452786528036, finishTime=1452786530568, timeTaken=2532, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=485490688, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786530556, Process_Duration=6, Process_End_Time=1452786530562, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786530511, firstTasksToStart=[ task_1452786527849_0001_1_01_000000 ], lastTaskFinishTime=1452786530566, lastTasksToFinish=[ task_1452786527849_0001_1_01_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786527849_0001_1_01_000000 ], longestDurationTasks=[ task_1452786527849_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:50,569 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:50,569 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:50,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786527849_0001_00_000002, stoppedTime=1452786530569, exitStatus=0
2016-01-14 15:48:50,570 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:50,570 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:50,570 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786527849_0001_1
2016-01-14 15:48:50,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786527849_0001_1
2016-01-14 15:48:50,570 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3F8A50AC3C3B43C7AB05F2AB25B567B4 for vertex: vertex_1452786527849_0001_1_01 [DACF065D8A0D45B081A85ED51CC147CD]
2016-01-14 15:48:50,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:50,570 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786527849_0001_00_000002
2016-01-14 15:48:50,571 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786527849_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:50,571 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786527849_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:50,571 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786527849_0001_00_000002, Exiting
2016-01-14 15:48:50,571 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786527849_0001_00_000002
2016-01-14 15:48:50,571 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786527849_0001_00_000002
2016-01-14 15:48:50,572 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786527849_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:50,572 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786527849_0001_00_000002 completed successfully
2016-01-14 15:48:50,579 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786527849_0001_1_00 [D868CBC29DA640CBB3237CBE9102B94D]
2016-01-14 15:48:50,581 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786527849_0001_1, queueSize=0
2016-01-14 15:48:50,582 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:50,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786527849_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786527849_0001_1, startTime=1452786528010, finishTime=1452786530579, timeTaken=2569, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=339, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=970981376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573058649, Process_Duration=11, Process_End_Time=2905573058660, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:50,582 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786527849_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:50,582 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786527849_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:50,583 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786527849_0001_00_000002
2016-01-14 15:48:50,583 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786527849_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:50 Completed Dag: dag_1452786527849_0001_1
2016-01-14 15:48:50,584 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786527849_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:50,584 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786527849_0001_1_post
2016-01-14 15:48:50,584 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:50,584 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:50,587 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_false_true_false, applicationId=application_1452786527849_0001
2016-01-14 15:48:50,589 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:50,589 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:50,589 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:50,589 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:50,589 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:50,590 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:50,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:50,592 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:50,592 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:50,592 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:50,593 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:50,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:50,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:50,599 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36087F601079492BB9FD86E2C43458CF/F76C27EEF91F4DF09FFEC33BC0DFE37E/.tez/application_1452786527849_0001
2016-01-14 15:48:50,600 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:02.767
2016-01-14 15:48:50,653 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:50,654 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:50,683 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 15:48:50,692 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:48:50,692 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:50,692 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:50,693 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:50,704 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:50,715 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false
2016-01-14 15:48:50,759 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:50,759 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:50,759 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false"]
2016-01-14 15:48:50,759 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:50,760 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:50,760 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:50,760 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...sive/true_true_true_false
2016-01-14 15:48:50,762 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:50,771 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:50,776 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:50,776 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:50,776 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:50,785 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747/.tez/application_1452786530776_0001 doesn't exist and is created
2016-01-14 15:48:50,800 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:50,800 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:50,801 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747/.tez/application_1452786530776_0001
2016-01-14 15:48:50,806 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786530776_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:50,814 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:50,815 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:50,817 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:50,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:50,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747/.tez/application_1452786530776_0001/localmode-log-dir
2016-01-14 15:48:50,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747/.tez/application_1452786530776_0001/localmode-log-dir/history.txt.appattempt_1452786530776_0001_000000, maxErrors=10
2016-01-14 15:48:50,818 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:50,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786530776_0001_000000, appSubmitTime=1452786530804, launchTime=1452786530804
2016-01-14 15:48:50,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:50,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:50,823 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:50,823 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:50,825 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55897
2016-01-14 15:48:50,827 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:50,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786530776_0001_000000, startTime=1452786530829
2016-01-14 15:48:50,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:50,829 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:50,901 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:50,901 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:50,901 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_true_true_false, applicationId=application_1452786530776_0001, dagName=[03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747] comprehensivesort/(1/1) ...sive/true_true_true_false
2016-01-14 15:48:50,914 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747] comprehensivesort/(1/1) ...sive/true_true_true_false
2016-01-14 15:48:50,916 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786530776_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747/.tez/application_1452786530776_0001/localmode-log-dir/dag_1452786530776_0001_1.dot
2016-01-14 15:48:50,916 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747/.tez/application_1452786530776_0001/localmode-log-dir/dag_1452786530776_0001_1-tez-dag.pb.txt
2016-01-14 15:48:50,925 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786530776_0001_1
2016-01-14 15:48:50,925 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:50,925 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747] comprehensivesort/(1/1) ...sive/true_true_true_false
2016-01-14 15:48:50 Running Dag: dag_1452786530776_0001_1
2016-01-14 15:48:50,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786530776_0001_1, submitTime=1452786530914
2016-01-14 15:48:50,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:50,937 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EB5AE052C81A4BC4A0692D267B82F2EF
2016-01-14 15:48:50,937 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:50,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786530776_0001_1, initTime=1452786530937
2016-01-14 15:48:50,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:50,938 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786530776_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:50,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:DAG_STARTED]: dagID=dag_1452786530776_0001_1, startTime=1452786530938
2016-01-14 15:48:50,939 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:50,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:50,939 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786530776_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:50,939 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_true_true_false, applicationId=application_1452786530776_0001, dagName=[03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747] comprehensivesort/(1/1) ...sive/true_true_true_false
2016-01-14 15:48:50,939 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CF9CD19A8F77452689D373ACCB49E7C6 : {30FF1AB98B434770BD060AD1E74ECEB0={InputName=30FF1AB98B434770BD060AD1E74ECEB0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:50,939 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 30FF1AB98B434770BD060AD1E74ECEB0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:50,940 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:50,940 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:50,940 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:50,940 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]. Starting root input initializers: 1
2016-01-14 15:48:50,940 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:50,941 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:50,941 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 30FF1AB98B434770BD060AD1E74ECEB0 on vertex vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:50,942 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:50,943 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 30FF1AB98B434770BD060AD1E74ECEB0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:50,943 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:50,943 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:50,944 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:50,944 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:50,944 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:50,944 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:50,946 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C8C82FF5D41640C98B81EE79B70070BF, vertexId=vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:50,945 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:50,946 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C8C82FF5D41640C98B81EE79B70070BF, vertexId=vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:50,947 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 30FF1AB98B434770BD060AD1E74ECEB0 on vertex vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:50,948 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C8C82FF5D41640C98B81EE79B70070BF, vertexName=EB5AE052C81A4BC4A0692D267B82F2EF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:50,949 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C8C82FF5D41640C98B81EE79B70070BF, vertexId=vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:50,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:50,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EB5AE052C81A4BC4A0692D267B82F2EF, vertexId=vertex_1452786530776_0001_1_01, initRequestedTime=1452786530941, initedTime=1452786530950, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:50,950 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:50,950 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:50,950 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {30FF1AB98B434770BD060AD1E74ECEB0=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:50,951 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] parallelism set to 1
2016-01-14 15:48:50,951 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:50,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786530776_0001_1_00, eventCount=1
2016-01-14 15:48:50,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CF9CD19A8F77452689D373ACCB49E7C6, vertexId=vertex_1452786530776_0001_1_00, initRequestedTime=1452786530939, initedTime=1452786530952, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:50,952 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] with distanceFromRoot: 0
2016-01-14 15:48:50,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:50,952 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:50,953 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:50,953 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:50,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:50,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786530776_0001_1_00, startRequestedTime=1452786530953, startedTime=1452786530953
2016-01-14 15:48:50,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:50,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:50,954 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786530776_0001, with dag id: dag_1452786530776_0001_1
2016-01-14 15:48:50,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:TASK_STARTED]: vertexName=CF9CD19A8F77452689D373ACCB49E7C6, taskId=task_1452786530776_0001_1_00_000000, scheduledTime=1452786530954, launchTime=1452786530954
2016-01-14 15:48:50,956 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786530776_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:50,956 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786530776_0001_1_00 for vertex: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF] numStartedSources: 1 numSources: 1
2016-01-14 15:48:50,956 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF] with distanceFromRoot: 1
2016-01-14 15:48:50,957 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:50,958 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CF9CD19A8F77452689D373ACCB49E7C6 in vertex: EB5AE052C81A4BC4A0692D267B82F2EF
2016-01-14 15:48:50,958 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EB5AE052C81A4BC4A0692D267B82F2EF with 1 source tasks and 1 pending tasks
2016-01-14 15:48:50,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:50,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786530776_0001_1_01, startRequestedTime=1452786530957, startedTime=1452786530957
2016-01-14 15:48:50,961 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:50,961 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747] comprehensivesort/(1/1) ...sive/true_true_true_false, VertexName: CF9CD19A8F77452689D373ACCB49E7C6, VertexParallelism: 1, TaskAttemptID:attempt_1452786530776_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=30FF1AB98B434770BD060AD1E74ECEB0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EB5AE052C81A4BC4A0692D267B82F2EF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:50,962 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786530776_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:50,962 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:50,963 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:50,963 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:50,963 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:50,963 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786530776_0001_00_000001
2016-01-14 15:48:50,964 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:50,965 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786530776_0001_00_000001
2016-01-14 15:48:50,965 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:50,966 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:50,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:50,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786530776_0001_00_000001, launchTime=1452786530966
2016-01-14 15:48:50,966 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786530776_0001_00_000001
2016-01-14 15:48:50,967 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786530776_0001_1_00_000000_0] to container: [container_1452786530776_0001_00_000001]
2016-01-14 15:48:50,967 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:50,967 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786530776_0001_00_000001 given task: attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:50,967 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786530776_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:50,967 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786530776_0001_1_00_000000_0] started. Is using containerId: [container_1452786530776_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:50 Starting to run new task attempt: attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:50,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CF9CD19A8F77452689D373ACCB49E7C6, taskAttemptId=attempt_1452786530776_0001_1_00_000000_0, startTime=1452786530967, containerId=container_1452786530776_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786530776_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:50,968 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786530776_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:50,968 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:50,968 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786530776_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:50,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:50,968 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:50,968 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:50,968 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:50,968 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:50,969 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:50,969 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:50,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747] comprehensivesort/(1/1) ...sive/true_true_true_false, VertexName: CF9CD19A8F77452689D373ACCB49E7C6, VertexParallelism: 1, TaskAttemptID:attempt_1452786530776_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=30FF1AB98B434770BD060AD1E74ECEB0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EB5AE052C81A4BC4A0692D267B82F2EF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:50,969 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:50,969 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:50,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:50,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:50,970 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786530776_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:50,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=30FF1AB98B434770BD060AD1E74ECEB0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:50,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EB5AE052C81A4BC4A0692D267B82F2EF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:50,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:50,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:50,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 30FF1AB98B434770BD060AD1E74ECEB0
2016-01-14 15:48:50,971 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 30FF1AB98B434770BD060AD1E74ECEB0
2016-01-14 15:48:50,973 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:50,974 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:50,974 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 30FF1AB98B434770BD060AD1E74ECEB0
2016-01-14 15:48:50,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 30FF1AB98B434770BD060AD1E74ECEB0
2016-01-14 15:48:50,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:50,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EB5AE052C81A4BC4A0692D267B82F2EF
2016-01-14 15:48:50,977 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:50,977 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EB5AE052C81A4BC4A0692D267B82F2EF
2016-01-14 15:48:50,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EB5AE052C81A4BC4A0692D267B82F2EF
2016-01-14 15:48:50,980 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CF9CD19A8F77452689D373ACCB49E7C6, ordinal: 0
2016-01-14 15:48:50,980 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CF9CD19A8F77452689D373ACCB49E7C6, mem on start (mb), free: 120, total: 463, max: 672
2016-01-14 15:48:50,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:50,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:50,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:50,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:50,980 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:50,980 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:50,980 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:50,981 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:50,981 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:50,981 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:50,981 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 30FF1AB98B434770BD060AD1E74ECEB0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:50,981 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EB5AE052C81A4BC4A0692D267B82F2EF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:50,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:50,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 30FF1AB98B434770BD060AD1E74ECEB0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:50,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 30FF1AB98B434770BD060AD1E74ECEB0
2016-01-14 15:48:50,982 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:50,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 30FF1AB98B434770BD060AD1E74ECEB0
2016-01-14 15:48:50,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:50,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:50,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:50,982 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:50,983 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:50,983 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4ece4cd0
2016-01-14 15:48:50,984 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4ece4cd0
2016-01-14 15:48:50,984 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:50,984 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:50,985 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:50,985 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 30FF1AB98B434770BD060AD1E74ECEB0
2016-01-14 15:48:50,985 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: B6EEB4566143456C85BC20EABB1FC19C
2016-01-14 15:48:50,986 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] B6EEB4566143456C85BC20EABB1FC19C
2016-01-14 15:48:50,988 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@1c2920f2; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5031fe4f; comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@6344c6ab; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:50,988 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:50,988 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:50,988 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:51,010 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:51,010 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:51,010 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:51,010 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:51,010 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:51,011 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:51,011 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:51,011 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CF9CD19A8F77452689D373ACCB49E7C6, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:51,016 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:51,016 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CF9CD19A8F77452689D373ACCB49E7C6, mem on close (mb), free: 215, total: 458, max: 672
2016-01-14 15:48:51,016 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:51,016 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:51,016 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:51,016 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:52,125 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:52,125 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:52,671 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:52,671 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:53,892 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:53 Completed running task attempt: attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:53,896 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786530776_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:53,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786530776_0001_1_00, eventCount=1
2016-01-14 15:48:53,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=361, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=493355008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786531011, Process_Duration=5, Process_End_Time=1452786531016, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:53,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CF9CD19A8F77452689D373ACCB49E7C6, taskAttemptId=attempt_1452786530776_0001_1_00_000000_0, startTime=1452786530967, finishTime=1452786533899, timeTaken=2932, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=361, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=493355008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786531011, Process_Duration=5, Process_End_Time=1452786531016, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:53,901 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:53,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786530776_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:53,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:53,901 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786530776_0001_1_00_000000_0
2016-01-14 15:48:53,901 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:53,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:53,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:TASK_FINISHED]: vertexName=CF9CD19A8F77452689D373ACCB49E7C6, taskId=task_1452786530776_0001_1_00_000000, startTime=1452786530967, finishTime=1452786533901, timeTaken=2934, status=SUCCEEDED, successfulAttemptID=attempt_1452786530776_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=361, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=493355008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786531011, Process_Duration=5, Process_End_Time=1452786531016, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:53,903 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786530776_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:53,903 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] : 1
2016-01-14 15:48:53,903 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:53,904 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:53,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:VERTEX_FINISHED]: vertexName=CF9CD19A8F77452689D373ACCB49E7C6, vertexId=vertex_1452786530776_0001_1_00, initRequestedTime=1452786530939, initedTime=1452786530952, startRequestedTime=1452786530953, startedTime=1452786530953, finishTime=1452786533903, timeTaken=2950, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=361, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=493355008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786531011, Process_Duration=5, Process_End_Time=1452786531016, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786530967, firstTasksToStart=[ task_1452786530776_0001_1_00_000000 ], lastTaskFinishTime=1452786533899, lastTasksToFinish=[ task_1452786530776_0001_1_00_000000 ], minTaskDuration=2932, maxTaskDuration=2932, avgTaskDuration=2932.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786530776_0001_1_00_000000 ], longestDurationTasks=[ task_1452786530776_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:53,904 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:53,904 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:53,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:53,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786530776_0001_00_000001, stoppedTime=1452786533905, exitStatus=0
2016-01-14 15:48:53,905 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:53,905 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF] attempt: attempt_1452786530776_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:53,905 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EB5AE052C81A4BC4A0692D267B82F2EF
2016-01-14 15:48:53,905 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:53,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:53,906 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:53,906 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786530776_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:53,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:TASK_STARTED]: vertexName=EB5AE052C81A4BC4A0692D267B82F2EF, taskId=task_1452786530776_0001_1_01_000000, scheduledTime=1452786533906, launchTime=1452786533906
2016-01-14 15:48:53,906 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:53,907 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786530776_0001_00_000001
2016-01-14 15:48:53,907 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786530776_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:53,907 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786530776_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:53,906 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786530776_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:53,907 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786530776_0001_00_000001, Exiting
2016-01-14 15:48:53,908 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786530776_0001_00_000001
2016-01-14 15:48:53,908 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786530776_0001_00_000001
2016-01-14 15:48:53,908 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:53,911 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:53,911 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747] comprehensivesort/(1/1) ...sive/true_true_true_false, VertexName: EB5AE052C81A4BC4A0692D267B82F2EF, VertexParallelism: 1, TaskAttemptID:attempt_1452786530776_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CF9CD19A8F77452689D373ACCB49E7C6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C8C82FF5D41640C98B81EE79B70070BF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:53,911 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786530776_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:53,911 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:53,912 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786530776_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:53,912 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786530776_0001_00_000001 completed successfully
2016-01-14 15:48:53,912 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786530776_0001_00_000002
2016-01-14 15:48:53,913 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:53,914 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:53,915 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786530776_0001_00_000002
2016-01-14 15:48:53,915 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:53,917 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:53,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:53,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786530776_0001_00_000002, launchTime=1452786533917
2016-01-14 15:48:53,918 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786530776_0001_00_000002
2016-01-14 15:48:53,919 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786530776_0001_1_01_000000_0] to container: [container_1452786530776_0001_00_000002]
2016-01-14 15:48:53,919 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:53,919 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786530776_0001_1_01_000000_0] started. Is using containerId: [container_1452786530776_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:53,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EB5AE052C81A4BC4A0692D267B82F2EF, taskAttemptId=attempt_1452786530776_0001_1_01_000000_0, startTime=1452786533919, containerId=container_1452786530776_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786530776_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:53,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:53,919 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786530776_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:53,920 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786530776_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:53,920 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786530776_0001_00_000002 given task: attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,920 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786530776_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53 Starting to run new task attempt: attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,920 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,920 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:53,921 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:53,921 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:53,921 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:53,921 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:53,921 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:53,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747] comprehensivesort/(1/1) ...sive/true_true_true_false, VertexName: EB5AE052C81A4BC4A0692D267B82F2EF, VertexParallelism: 1, TaskAttemptID:attempt_1452786530776_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CF9CD19A8F77452689D373ACCB49E7C6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C8C82FF5D41640C98B81EE79B70070BF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:53,922 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:53,922 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:53,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:53,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:53,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C8C82FF5D41640C98B81EE79B70070BF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:53,924 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786530776_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:53,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:53,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C8C82FF5D41640C98B81EE79B70070BF
2016-01-14 15:48:53,924 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:53,924 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C8C82FF5D41640C98B81EE79B70070BF
2016-01-14 15:48:53,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CF9CD19A8F77452689D373ACCB49E7C6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:53,926 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:53,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:53,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:53,927 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:53,927 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:53,928 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:53,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:53,933 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EB5AE052C81A4BC4A0692D267B82F2EF, ordinal: 1
2016-01-14 15:48:53,933 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EB5AE052C81A4BC4A0692D267B82F2EF, mem on start (mb), free: 221, total: 470, max: 672
2016-01-14 15:48:53,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:53,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:53,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:53,936 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:53,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C8C82FF5D41640C98B81EE79B70070BF
2016-01-14 15:48:53,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:53,936 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:53,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:53,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:53,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:53,937 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:53,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:53,938 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C8C82FF5D41640C98B81EE79B70070BF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:53,938 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CF9CD19A8F77452689D373ACCB49E7C6, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:53,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:53,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CF9CD19A8F77452689D373ACCB49E7C6 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:53,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:53,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:53,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:53,939 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:53,939 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:53,939 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CF9CD19A8F77452689D373ACCB49E7C6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:53,940 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:53,940 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:53,940 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:53,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:53,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:53,941 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,943 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786530776_0001_1_00_000000_0_10083, runDuration: 2919241, ]
2016-01-14 15:48:53,944 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786530776_0001_1_00_000000_0_10083] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786533944, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:53,944 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:53,944 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:53,944 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:53,945 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CF9CD19A8F77452689D373ACCB49E7C6] #15 in 2ms
2016-01-14 15:48:53,945 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: B6EEB4566143456C85BC20EABB1FC19C
2016-01-14 15:48:53,945 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CF9CD19A8F77452689D373ACCB49E7C6] #15, status:false, isInterrupted:false
2016-01-14 15:48:53,945 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false"], id: C8C82FF5D41640C98B81EE79B70070BF
2016-01-14 15:48:53,946 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false"]
2016-01-14 15:48:53,946 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] B6EEB4566143456C85BC20EABB1FC19C, for 1 inputs
2016-01-14 15:48:53,946 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:53,948 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:53,948 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:53,949 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:53,949 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:53,949 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EB5AE052C81A4BC4A0692D267B82F2EF, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:48:53,949 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:53,951 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:53,951 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@778a7d6; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@252d66f
2016-01-14 15:48:53,955 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:53,955 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:53,956 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,956 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786530776_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:53,956 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,957 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EB5AE052C81A4BC4A0692D267B82F2EF, mem on close (mb), free: 217, total: 470, max: 672
2016-01-14 15:48:53,957 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CF9CD19A8F77452689D373ACCB49E7C6
2016-01-14 15:48:53,957 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:53,959 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:53,959 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:53,959 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:53,959 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:53 Completed running task attempt: attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,960 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786530776_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:53,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=493355008, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786533949, Process_Duration=7, Process_End_Time=1452786533956, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:53,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EB5AE052C81A4BC4A0692D267B82F2EF, taskAttemptId=attempt_1452786530776_0001_1_01_000000_0, startTime=1452786533919, finishTime=1452786533961, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=493355008, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786533949, Process_Duration=7, Process_End_Time=1452786533956, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:53,961 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:53,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:53,962 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:53,961 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786530776_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:53,962 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786530776_0001_00_000002
2016-01-14 15:48:53,962 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786530776_0001_1_01_000000_0
2016-01-14 15:48:53,962 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:53,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:53,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:TASK_FINISHED]: vertexName=EB5AE052C81A4BC4A0692D267B82F2EF, taskId=task_1452786530776_0001_1_01_000000, startTime=1452786533919, finishTime=1452786533962, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452786530776_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=493355008, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786533949, Process_Duration=7, Process_End_Time=1452786533956, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:53,963 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786530776_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:53,963 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF] : 1
2016-01-14 15:48:53,963 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:53,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:VERTEX_FINISHED]: vertexName=EB5AE052C81A4BC4A0692D267B82F2EF, vertexId=vertex_1452786530776_0001_1_01, initRequestedTime=1452786530941, initedTime=1452786530950, startRequestedTime=1452786530957, startedTime=1452786530957, finishTime=1452786533963, timeTaken=3006, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=493355008, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786533949, Process_Duration=7, Process_End_Time=1452786533956, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786533919, firstTasksToStart=[ task_1452786530776_0001_1_01_000000 ], lastTaskFinishTime=1452786533961, lastTasksToFinish=[ task_1452786530776_0001_1_01_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786530776_0001_1_01_000000 ], longestDurationTasks=[ task_1452786530776_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:53,964 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:53,964 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:53,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786530776_0001_00_000002, stoppedTime=1452786533964, exitStatus=0
2016-01-14 15:48:53,964 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:53,964 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:53,965 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786530776_0001_1
2016-01-14 15:48:53,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786530776_0001_1
2016-01-14 15:48:53,965 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786530776_0001_1_00 [CF9CD19A8F77452689D373ACCB49E7C6]
2016-01-14 15:48:53,965 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C8C82FF5D41640C98B81EE79B70070BF for vertex: vertex_1452786530776_0001_1_01 [EB5AE052C81A4BC4A0692D267B82F2EF]
2016-01-14 15:48:53,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:53,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:53,972 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786530776_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:53,972 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786530776_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:53,973 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786530776_0001_00_000002, Exiting
2016-01-14 15:48:53,973 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786530776_0001_00_000002
2016-01-14 15:48:53,973 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786530776_0001_00_000002
2016-01-14 15:48:53,973 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786530776_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:53,974 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786530776_0001_00_000002 completed successfully
2016-01-14 15:48:53,974 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786530776_0001_1, queueSize=0
2016-01-14 15:48:53,974 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:53,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786530776_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786530776_0001_1, startTime=1452786530938, finishTime=1452786533971, timeTaken=3033, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=361, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=986710016, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573064960, Process_Duration=12, Process_End_Time=2905573064972, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:53,975 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786530776_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:53,975 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786530776_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:53,976 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786530776_0001_00_000002
2016-01-14 15:48:53,976 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786530776_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:53 Completed Dag: dag_1452786530776_0001_1
2016-01-14 15:48:53,977 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786530776_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:53,977 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786530776_0001_1_post
2016-01-14 15:48:53,977 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:53,977 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:53,980 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_true_true_false, applicationId=application_1452786530776_0001
2016-01-14 15:48:53,981 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:53,982 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:53,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:53,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:53,982 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:53,982 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:53,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:53,983 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:53,983 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:53,983 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:53,984 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:53,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:53,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:53,988 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/03AF9FE4BFAB4B81BC669B56044AC9C7/C4CF410251BD4BD3924654CFBDA9C747/.tez/application_1452786530776_0001
2016-01-14 15:48:53,988 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:03.229
2016-01-14 15:48:54,041 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:54,042 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:54,076 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 15:48:54,082 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:48:54,083 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:54,083 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:54,083 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:54,096 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:54,103 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false
2016-01-14 15:48:54,140 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:54,141 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:54,141 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false"]
2016-01-14 15:48:54,141 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:54,142 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:54,142 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:54,143 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/false_true_true_false
2016-01-14 15:48:54,143 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:54,151 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:54,156 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:54,156 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:54,156 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:54,165 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77/.tez/application_1452786534156_0001 doesn't exist and is created
2016-01-14 15:48:54,180 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:54,180 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:54,181 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77/.tez/application_1452786534156_0001
2016-01-14 15:48:54,184 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786534156_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:54,192 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:54,192 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:54,194 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:54,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:54,194 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77/.tez/application_1452786534156_0001/localmode-log-dir
2016-01-14 15:48:54,194 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77/.tez/application_1452786534156_0001/localmode-log-dir/history.txt.appattempt_1452786534156_0001_000000, maxErrors=10
2016-01-14 15:48:54,194 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:54,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786534156_0001_000000, appSubmitTime=1452786534183, launchTime=1452786534183
2016-01-14 15:48:54,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:54,198 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:54,198 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:54,199 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:54,200 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60655
2016-01-14 15:48:54,202 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:54,204 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786534156_0001_000000, startTime=1452786534204
2016-01-14 15:48:54,204 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:54,204 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:54,281 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:54,281 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:54,281 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_true_true_false, applicationId=application_1452786534156_0001, dagName=[2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77] comprehensivesort/(1/1) ...ive/false_true_true_false
2016-01-14 15:48:54,293 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77] comprehensivesort/(1/1) ...ive/false_true_true_false
2016-01-14 15:48:54,294 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786534156_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77/.tez/application_1452786534156_0001/localmode-log-dir/dag_1452786534156_0001_1.dot
2016-01-14 15:48:54,295 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77/.tez/application_1452786534156_0001/localmode-log-dir/dag_1452786534156_0001_1-tez-dag.pb.txt
2016-01-14 15:48:54,307 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786534156_0001_1
2016-01-14 15:48:54,307 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:54,307 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77] comprehensivesort/(1/1) ...ive/false_true_true_false
2016-01-14 15:48:54 Running Dag: dag_1452786534156_0001_1
2016-01-14 15:48:54,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786534156_0001_1, submitTime=1452786534293
2016-01-14 15:48:54,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:54,318 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A0B99B20D95148BD8315732C1A85AFC1
2016-01-14 15:48:54,319 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:54,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786534156_0001_1, initTime=1452786534318
2016-01-14 15:48:54,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:54,320 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786534156_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:54,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:DAG_STARTED]: dagID=dag_1452786534156_0001_1, startTime=1452786534320
2016-01-14 15:48:54,321 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:54,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:54,321 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786534156_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:54,321 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AF134C30519F4B598F7055AC90B12537 : {702E00D380814D799AF28D8A1E44165C={InputName=702E00D380814D799AF28D8A1E44165C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:54,321 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 702E00D380814D799AF28D8A1E44165C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:54,321 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:54,322 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_true_true_false, applicationId=application_1452786534156_0001, dagName=[2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77] comprehensivesort/(1/1) ...ive/false_true_true_false
2016-01-14 15:48:54,322 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:54,322 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:54,322 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]. Starting root input initializers: 1
2016-01-14 15:48:54,323 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:54,323 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:54,323 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 702E00D380814D799AF28D8A1E44165C on vertex vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:54,325 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:54,325 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 702E00D380814D799AF28D8A1E44165C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:54,325 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:54,325 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:54,326 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:54,326 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:54,326 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:54,326 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:54,326 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=20A489F236CA404084C6060FFDADDAC7, vertexId=vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:54,327 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=20A489F236CA404084C6060FFDADDAC7, vertexId=vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:54,327 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:54,327 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 702E00D380814D799AF28D8A1E44165C on vertex vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:54,328 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=20A489F236CA404084C6060FFDADDAC7, vertexName=A0B99B20D95148BD8315732C1A85AFC1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:54,329 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=20A489F236CA404084C6060FFDADDAC7, vertexId=vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:54,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:54,330 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A0B99B20D95148BD8315732C1A85AFC1, vertexId=vertex_1452786534156_0001_1_01, initRequestedTime=1452786534323, initedTime=1452786534330, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:54,330 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:54,330 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:54,331 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {702E00D380814D799AF28D8A1E44165C=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:54,331 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] parallelism set to 1
2016-01-14 15:48:54,331 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:54,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786534156_0001_1_00, eventCount=1
2016-01-14 15:48:54,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AF134C30519F4B598F7055AC90B12537, vertexId=vertex_1452786534156_0001_1_00, initRequestedTime=1452786534321, initedTime=1452786534332, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:54,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:54,332 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] with distanceFromRoot: 0
2016-01-14 15:48:54,333 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:54,333 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:54,333 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:54,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786534156_0001_1_00, startRequestedTime=1452786534333, startedTime=1452786534333
2016-01-14 15:48:54,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:54,334 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:54,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:54,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:TASK_STARTED]: vertexName=AF134C30519F4B598F7055AC90B12537, taskId=task_1452786534156_0001_1_00_000000, scheduledTime=1452786534334, launchTime=1452786534334
2016-01-14 15:48:54,335 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786534156_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:54,335 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786534156_0001_1_00 for vertex: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1] numStartedSources: 1 numSources: 1
2016-01-14 15:48:54,335 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1] with distanceFromRoot: 1
2016-01-14 15:48:54,336 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:54,336 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AF134C30519F4B598F7055AC90B12537 in vertex: A0B99B20D95148BD8315732C1A85AFC1
2016-01-14 15:48:54,336 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A0B99B20D95148BD8315732C1A85AFC1 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:54,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786534156_0001_1_01, startRequestedTime=1452786534336, startedTime=1452786534336
2016-01-14 15:48:54,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:54,337 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:54,337 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786534156_0001, with dag id: dag_1452786534156_0001_1
2016-01-14 15:48:54,337 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77] comprehensivesort/(1/1) ...ive/false_true_true_false, VertexName: AF134C30519F4B598F7055AC90B12537, VertexParallelism: 1, TaskAttemptID:attempt_1452786534156_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=702E00D380814D799AF28D8A1E44165C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0B99B20D95148BD8315732C1A85AFC1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:54,338 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786534156_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:54,338 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:54,340 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:54,340 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:54,340 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:54,341 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786534156_0001_00_000001
2016-01-14 15:48:54,341 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:54,342 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786534156_0001_00_000001
2016-01-14 15:48:54,342 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:54,343 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:54,343 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:54,343 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786534156_0001_00_000001, launchTime=1452786534343
2016-01-14 15:48:54,343 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786534156_0001_00_000001
2016-01-14 15:48:54,344 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786534156_0001_1_00_000000_0] to container: [container_1452786534156_0001_00_000001]
2016-01-14 15:48:54,344 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:54,344 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786534156_0001_00_000001 given task: attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:54,344 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786534156_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:54,344 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786534156_0001_1_00_000000_0] started. Is using containerId: [container_1452786534156_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:54,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AF134C30519F4B598F7055AC90B12537, taskAttemptId=attempt_1452786534156_0001_1_00_000000_0, startTime=1452786534344, containerId=container_1452786534156_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786534156_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:54,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:54 Starting to run new task attempt: attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:54,345 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:54,345 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:54,345 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786534156_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:54,345 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:54,345 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786534156_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:54,345 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:54,345 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:54,346 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:54,346 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:54,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77] comprehensivesort/(1/1) ...ive/false_true_true_false, VertexName: AF134C30519F4B598F7055AC90B12537, VertexParallelism: 1, TaskAttemptID:attempt_1452786534156_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=702E00D380814D799AF28D8A1E44165C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0B99B20D95148BD8315732C1A85AFC1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:54,346 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:54,346 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:54,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:54,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:54,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:54,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=702E00D380814D799AF28D8A1E44165C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:54,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:54,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 702E00D380814D799AF28D8A1E44165C
2016-01-14 15:48:54,349 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 702E00D380814D799AF28D8A1E44165C
2016-01-14 15:48:54,351 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:54,351 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:54,351 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786534156_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:54,352 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 702E00D380814D799AF28D8A1E44165C
2016-01-14 15:48:54,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 702E00D380814D799AF28D8A1E44165C
2016-01-14 15:48:54,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A0B99B20D95148BD8315732C1A85AFC1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:54,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:54,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A0B99B20D95148BD8315732C1A85AFC1
2016-01-14 15:48:54,354 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:54,354 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A0B99B20D95148BD8315732C1A85AFC1
2016-01-14 15:48:54,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A0B99B20D95148BD8315732C1A85AFC1
2016-01-14 15:48:54,359 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AF134C30519F4B598F7055AC90B12537, ordinal: 0
2016-01-14 15:48:54,359 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF134C30519F4B598F7055AC90B12537, mem on start (mb), free: 125, total: 470, max: 672
2016-01-14 15:48:54,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:54,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:54,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:54,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:54,360 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:54,360 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:54,360 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:54,360 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:54,360 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:54,360 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:54,360 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 702E00D380814D799AF28D8A1E44165C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:54,360 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A0B99B20D95148BD8315732C1A85AFC1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:54,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:54,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 702E00D380814D799AF28D8A1E44165C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:54,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:54,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:54,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 702E00D380814D799AF28D8A1E44165C
2016-01-14 15:48:54,361 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:54,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 702E00D380814D799AF28D8A1E44165C
2016-01-14 15:48:54,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:54,362 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:54,363 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:54,363 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@647ceedf
2016-01-14 15:48:54,364 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:54,365 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 702E00D380814D799AF28D8A1E44165C
2016-01-14 15:48:54,365 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: AB07C697A4DB494B83A1F3EF3D265C7F
2016-01-14 15:48:54,365 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] AB07C697A4DB494B83A1F3EF3D265C7F
2016-01-14 15:48:54,366 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@647ceedf
2016-01-14 15:48:54,366 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:54,366 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:54,367 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@5c31035d; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@15d3fdf4; comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@10c5a95e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:54,367 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:54,368 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:54,368 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:54,387 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:54,387 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:54,387 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:54,387 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:54,388 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:54,389 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:54,389 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:54,389 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AF134C30519F4B598F7055AC90B12537, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:54,394 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:54,394 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF134C30519F4B598F7055AC90B12537, mem on close (mb), free: 225, total: 471, max: 672
2016-01-14 15:48:54,394 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:54,395 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:54,395 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:54,395 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:55,590 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:55,590 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:55,888 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:55 Completed running task attempt: attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:55,891 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786534156_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:55,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=332, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=498073600, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786534390, Process_Duration=4, Process_End_Time=1452786534394, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:55,892 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:55,892 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:55,892 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786534156_0001_00_000001
2016-01-14 15:48:55,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786534156_0001_1_00, eventCount=1
2016-01-14 15:48:55,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AF134C30519F4B598F7055AC90B12537, taskAttemptId=attempt_1452786534156_0001_1_00_000000_0, startTime=1452786534344, finishTime=1452786535893, timeTaken=1549, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=332, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=498073600, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786534390, Process_Duration=4, Process_End_Time=1452786534394, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:55,893 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786534156_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:55,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:55,894 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786534156_0001_1_00_000000_0
2016-01-14 15:48:55,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:55,894 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:55,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:TASK_FINISHED]: vertexName=AF134C30519F4B598F7055AC90B12537, taskId=task_1452786534156_0001_1_00_000000, startTime=1452786534344, finishTime=1452786535894, timeTaken=1550, status=SUCCEEDED, successfulAttemptID=attempt_1452786534156_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=332, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=498073600, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786534390, Process_Duration=4, Process_End_Time=1452786534394, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:55,895 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786534156_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:55,895 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] : 1
2016-01-14 15:48:55,895 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:55,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:VERTEX_FINISHED]: vertexName=AF134C30519F4B598F7055AC90B12537, vertexId=vertex_1452786534156_0001_1_00, initRequestedTime=1452786534321, initedTime=1452786534332, startRequestedTime=1452786534333, startedTime=1452786534333, finishTime=1452786535895, timeTaken=1562, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=332, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=498073600, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786534390, Process_Duration=4, Process_End_Time=1452786534394, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786534344, firstTasksToStart=[ task_1452786534156_0001_1_00_000000 ], lastTaskFinishTime=1452786535893, lastTasksToFinish=[ task_1452786534156_0001_1_00_000000 ], minTaskDuration=1549, maxTaskDuration=1549, avgTaskDuration=1549.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786534156_0001_1_00_000000 ], longestDurationTasks=[ task_1452786534156_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:55,896 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:55,897 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:55,897 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1] attempt: attempt_1452786534156_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:55,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:55,897 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A0B99B20D95148BD8315732C1A85AFC1
2016-01-14 15:48:55,897 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:55,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786534156_0001_00_000001, stoppedTime=1452786535897, exitStatus=0
2016-01-14 15:48:55,897 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:55,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:55,898 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:55,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:TASK_STARTED]: vertexName=A0B99B20D95148BD8315732C1A85AFC1, taskId=task_1452786534156_0001_1_01_000000, scheduledTime=1452786535898, launchTime=1452786535898
2016-01-14 15:48:55,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:55,898 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786534156_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:55,898 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:55,898 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786534156_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:55,899 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77] comprehensivesort/(1/1) ...ive/false_true_true_false, VertexName: A0B99B20D95148BD8315732C1A85AFC1, VertexParallelism: 1, TaskAttemptID:attempt_1452786534156_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF134C30519F4B598F7055AC90B12537, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=20A489F236CA404084C6060FFDADDAC7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:55,899 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786534156_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:55,899 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:55,899 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:55,900 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786534156_0001_00_000002
2016-01-14 15:48:55,900 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:55,901 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786534156_0001_00_000002
2016-01-14 15:48:55,901 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:55,902 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:55,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:55,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786534156_0001_00_000002, launchTime=1452786535902
2016-01-14 15:48:55,903 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786534156_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:55,903 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786534156_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:55,904 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786534156_0001_00_000001, Exiting
2016-01-14 15:48:55,904 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786534156_0001_00_000001
2016-01-14 15:48:55,904 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786534156_0001_00_000001
2016-01-14 15:48:55,903 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786534156_0001_00_000002
2016-01-14 15:48:55,905 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786534156_0001_1_01_000000_0] to container: [container_1452786534156_0001_00_000002]
2016-01-14 15:48:55,905 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:55,905 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786534156_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:55,906 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786534156_0001_1_01_000000_0] started. Is using containerId: [container_1452786534156_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:55,906 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786534156_0001_00_000002 given task: attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,906 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:55,906 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786534156_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A0B99B20D95148BD8315732C1A85AFC1, taskAttemptId=attempt_1452786534156_0001_1_01_000000_0, startTime=1452786535906, containerId=container_1452786534156_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786534156_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:55,907 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786534156_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:55,907 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786534156_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:55 Starting to run new task attempt: attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,907 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,907 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:55,907 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:55,907 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:55,907 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:55,908 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786534156_0001_00_000001 completed successfully
2016-01-14 15:48:55,908 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:55,908 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:55,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77] comprehensivesort/(1/1) ...ive/false_true_true_false, VertexName: A0B99B20D95148BD8315732C1A85AFC1, VertexParallelism: 1, TaskAttemptID:attempt_1452786534156_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF134C30519F4B598F7055AC90B12537, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=20A489F236CA404084C6060FFDADDAC7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:55,908 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:55,908 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:55,909 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:55,909 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786534156_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:55,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:55,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AF134C30519F4B598F7055AC90B12537, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:55,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=20A489F236CA404084C6060FFDADDAC7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:55,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:55,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:55,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:55,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:55,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 20A489F236CA404084C6060FFDADDAC7
2016-01-14 15:48:55,913 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:55,913 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:55,914 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:55,914 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:55,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:55,914 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 20A489F236CA404084C6060FFDADDAC7
2016-01-14 15:48:55,914 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:55,921 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:55,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 20A489F236CA404084C6060FFDADDAC7
2016-01-14 15:48:55,922 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A0B99B20D95148BD8315732C1A85AFC1, ordinal: 1
2016-01-14 15:48:55,922 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0B99B20D95148BD8315732C1A85AFC1, mem on start (mb), free: 165, total: 475, max: 672
2016-01-14 15:48:55,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:55,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:55,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:55,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:55,923 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:55,923 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:55,923 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:55,923 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:55,923 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:55,924 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:55,924 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AF134C30519F4B598F7055AC90B12537, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:55,924 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 20A489F236CA404084C6060FFDADDAC7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:55,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:55,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:55,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AF134C30519F4B598F7055AC90B12537 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:55,925 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:55,926 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:55,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:55,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:55,926 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AF134C30519F4B598F7055AC90B12537 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:55,927 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:55,927 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:55,927 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:55,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:55,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:55,929 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,930 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786534156_0001_1_00_000000_0_10089, runDuration: 1536967, ]
2016-01-14 15:48:55,932 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:55,932 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786534156_0001_1_00_000000_0_10089] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786535932, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:55,932 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:55,932 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: AB07C697A4DB494B83A1F3EF3D265C7F
2016-01-14 15:48:55,932 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false"], id: 20A489F236CA404084C6060FFDADDAC7
2016-01-14 15:48:55,933 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false"]
2016-01-14 15:48:55,933 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:55,933 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] AB07C697A4DB494B83A1F3EF3D265C7F, for 1 inputs
2016-01-14 15:48:55,933 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AF134C30519F4B598F7055AC90B12537] #16 in 2ms
2016-01-14 15:48:55,934 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AF134C30519F4B598F7055AC90B12537] #16, status:false, isInterrupted:false
2016-01-14 15:48:55,934 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:55,935 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:55,935 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:55,935 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:55,935 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:55,935 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:55,935 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:55,935 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A0B99B20D95148BD8315732C1A85AFC1, all 1 inputs ready in: 00:00:00.002
2016-01-14 15:48:55,936 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@594abca7; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@39a7a5bd
2016-01-14 15:48:55,941 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:55,941 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:55,941 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,941 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786534156_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:55,942 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,942 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0B99B20D95148BD8315732C1A85AFC1, mem on close (mb), free: 161, total: 475, max: 672
2016-01-14 15:48:55,942 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AF134C30519F4B598F7055AC90B12537
2016-01-14 15:48:55,942 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:55,943 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:55,943 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:55,943 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:55,943 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:55 Completed running task attempt: attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,944 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786534156_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:55,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:55,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A0B99B20D95148BD8315732C1A85AFC1, taskAttemptId=attempt_1452786534156_0001_1_01_000000_0, startTime=1452786535906, finishTime=1452786535945, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=498073600, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786535935, Process_Duration=7, Process_End_Time=1452786535942, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:55,945 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786534156_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:55,946 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786534156_0001_1_01_000000_0
2016-01-14 15:48:55,946 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:55,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:55,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:TASK_FINISHED]: vertexName=A0B99B20D95148BD8315732C1A85AFC1, taskId=task_1452786534156_0001_1_01_000000, startTime=1452786535906, finishTime=1452786535946, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452786534156_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=498073600, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786535935, Process_Duration=7, Process_End_Time=1452786535942, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:55,946 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786534156_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:55,947 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1] : 1
2016-01-14 15:48:55,947 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:55,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:55,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:VERTEX_FINISHED]: vertexName=A0B99B20D95148BD8315732C1A85AFC1, vertexId=vertex_1452786534156_0001_1_01, initRequestedTime=1452786534323, initedTime=1452786534330, startRequestedTime=1452786534336, startedTime=1452786534336, finishTime=1452786535947, timeTaken=1611, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=498073600, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786535935, Process_Duration=7, Process_End_Time=1452786535942, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786535906, firstTasksToStart=[ task_1452786534156_0001_1_01_000000 ], lastTaskFinishTime=1452786535945, lastTasksToFinish=[ task_1452786534156_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786534156_0001_1_01_000000 ], longestDurationTasks=[ task_1452786534156_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:55,948 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:55,948 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:55,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786534156_0001_00_000002, stoppedTime=1452786535948, exitStatus=0
2016-01-14 15:48:55,949 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:55,949 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:55,949 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786534156_0001_1
2016-01-14 15:48:55,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:55,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786534156_0001_1
2016-01-14 15:48:55,950 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786534156_0001_1_00 [AF134C30519F4B598F7055AC90B12537]
2016-01-14 15:48:55,950 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 20A489F236CA404084C6060FFDADDAC7 for vertex: vertex_1452786534156_0001_1_01 [A0B99B20D95148BD8315732C1A85AFC1]
2016-01-14 15:48:55,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=498073600, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786535935, Process_Duration=7, Process_End_Time=1452786535942, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:55,950 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:55,950 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:55,951 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786534156_0001_00_000002
2016-01-14 15:48:55,951 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786534156_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:55,951 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786534156_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:55,951 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786534156_0001_00_000002, Exiting
2016-01-14 15:48:55,951 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786534156_0001_00_000002
2016-01-14 15:48:55,952 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786534156_0001_00_000002
2016-01-14 15:48:55,952 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786534156_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:55,952 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786534156_0001_00_000002 completed successfully
2016-01-14 15:48:55,959 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786534156_0001_1, queueSize=0
2016-01-14 15:48:55,959 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:55,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786534156_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786534156_0001_1, startTime=1452786534320, finishTime=1452786535957, timeTaken=1637, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=332, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=996147200, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573070325, Process_Duration=11, Process_End_Time=2905573070336, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:55,959 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786534156_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:55,960 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786534156_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:55,960 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786534156_0001_00_000002
2016-01-14 15:48:55,960 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786534156_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:55 Completed Dag: dag_1452786534156_0001_1
2016-01-14 15:48:55,961 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786534156_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:55,961 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786534156_0001_1_post
2016-01-14 15:48:55,961 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:55,961 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:55,964 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_true_true_false, applicationId=application_1452786534156_0001
2016-01-14 15:48:55,966 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:55,966 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:55,966 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:55,966 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:55,966 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:55,967 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:55,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:55,968 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:55,968 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:55,968 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:55,969 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:55,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:55,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:55,974 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2767D97C038548DD8E35028213924666/EB76B79607754CBC9B9E877AE0DB2D77/.tez/application_1452786534156_0001
2016-01-14 15:48:55,974 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:01.834
2016-01-14 15:48:56,025 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:56,025 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:56,056 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 15:48:56,070 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-01-14 15:48:56,070 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:56,071 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:56,071 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:56,082 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:56,088 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true
2016-01-14 15:48:56,122 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:56,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:56,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true"]
2016-01-14 15:48:56,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:56,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:56,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:56,124 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_false_true
2016-01-14 15:48:56,124 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:56,139 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:56,143 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:56,144 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:56,144 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:56,152 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A/.tez/application_1452786536144_0001 doesn't exist and is created
2016-01-14 15:48:56,167 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:56,167 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:56,168 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A/.tez/application_1452786536144_0001
2016-01-14 15:48:56,171 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786536144_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:56,179 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:56,180 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:56,181 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:56,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:56,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A/.tez/application_1452786536144_0001/localmode-log-dir
2016-01-14 15:48:56,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A/.tez/application_1452786536144_0001/localmode-log-dir/history.txt.appattempt_1452786536144_0001_000000, maxErrors=10
2016-01-14 15:48:56,182 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:56,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786536144_0001_000000, appSubmitTime=1452786536171, launchTime=1452786536171
2016-01-14 15:48:56,187 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59998
2016-01-14 15:48:56,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:56,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:56,191 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:56,191 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:56,192 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:56,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786536144_0001_000000, startTime=1452786536194
2016-01-14 15:48:56,194 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:56,194 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:56,268 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:56,268 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:56,268 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_null_false_true, applicationId=application_1452786536144_0001, dagName=[0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A] comprehensivesort/(1/1) ...ive/false_null_false_true
2016-01-14 15:48:56,280 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A] comprehensivesort/(1/1) ...ive/false_null_false_true
2016-01-14 15:48:56,281 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786536144_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A/.tez/application_1452786536144_0001/localmode-log-dir/dag_1452786536144_0001_1.dot
2016-01-14 15:48:56,282 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A/.tez/application_1452786536144_0001/localmode-log-dir/dag_1452786536144_0001_1-tez-dag.pb.txt
2016-01-14 15:48:56,289 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786536144_0001_1
2016-01-14 15:48:56,289 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,290 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A] comprehensivesort/(1/1) ...ive/false_null_false_true
2016-01-14 15:48:56 Running Dag: dag_1452786536144_0001_1
2016-01-14 15:48:56,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786536144_0001_1, submitTime=1452786536280
2016-01-14 15:48:56,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:56,301 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B4EB7529D81A42E1A9E50ECF80EBE390
2016-01-14 15:48:56,301 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:56,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786536144_0001_1, initTime=1452786536300
2016-01-14 15:48:56,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:56,302 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786536144_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:56,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:DAG_STARTED]: dagID=dag_1452786536144_0001_1, startTime=1452786536302
2016-01-14 15:48:56,303 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:56,303 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786536144_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:56,303 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C3DBBE829BFD4AECBFC408B6DF77E2BE : {2330E9FD883747EA9628137A87F90C09={InputName=2330E9FD883747EA9628137A87F90C09}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:56,303 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2330E9FD883747EA9628137A87F90C09, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:56,303 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:56,304 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,304 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,304 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]. Starting root input initializers: 1
2016-01-14 15:48:56,303 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_null_false_true, applicationId=application_1452786536144_0001, dagName=[0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A] comprehensivesort/(1/1) ...ive/false_null_false_true
2016-01-14 15:48:56,304 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:56,304 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,306 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:56,306 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,307 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,307 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,307 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B0B4E0F65D274D28BBC2A8964DBE12AE, vertexId=vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:56,307 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B0B4E0F65D274D28BBC2A8964DBE12AE, vertexId=vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,308 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B0B4E0F65D274D28BBC2A8964DBE12AE, vertexName=B4EB7529D81A42E1A9E50ECF80EBE390, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:56,308 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B0B4E0F65D274D28BBC2A8964DBE12AE, vertexId=vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,307 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2330E9FD883747EA9628137A87F90C09 on vertex vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:56,309 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B4EB7529D81A42E1A9E50ECF80EBE390, vertexId=vertex_1452786536144_0001_1_01, initRequestedTime=1452786536304, initedTime=1452786536309, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:56,312 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:56,315 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:56,315 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2330E9FD883747EA9628137A87F90C09 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:56,316 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:56,316 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:56,317 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:56,318 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2330E9FD883747EA9628137A87F90C09 on vertex vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,318 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786536144_0001, with dag id: dag_1452786536144_0001_1
2016-01-14 15:48:56,318 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:56,318 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2330E9FD883747EA9628137A87F90C09=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:56,319 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] parallelism set to 1
2016-01-14 15:48:56,319 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:56,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786536144_0001_1_00, eventCount=1
2016-01-14 15:48:56,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:56,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, vertexId=vertex_1452786536144_0001_1_00, initRequestedTime=1452786536303, initedTime=1452786536320, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:56,321 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] with distanceFromRoot: 0
2016-01-14 15:48:56,321 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:56,321 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,322 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,322 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:56,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786536144_0001_1_00, startRequestedTime=1452786536321, startedTime=1452786536321
2016-01-14 15:48:56,323 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:56,324 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:56,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:TASK_STARTED]: vertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, taskId=task_1452786536144_0001_1_00_000000, scheduledTime=1452786536323, launchTime=1452786536323
2016-01-14 15:48:56,324 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536144_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:56,324 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786536144_0001_1_00 for vertex: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390] numStartedSources: 1 numSources: 1
2016-01-14 15:48:56,324 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390] with distanceFromRoot: 1
2016-01-14 15:48:56,324 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,325 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C3DBBE829BFD4AECBFC408B6DF77E2BE in vertex: B4EB7529D81A42E1A9E50ECF80EBE390
2016-01-14 15:48:56,325 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B4EB7529D81A42E1A9E50ECF80EBE390 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:56,325 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:56,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786536144_0001_1_01, startRequestedTime=1452786536325, startedTime=1452786536325
2016-01-14 15:48:56,326 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:56,326 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A] comprehensivesort/(1/1) ...ive/false_null_false_true, VertexName: C3DBBE829BFD4AECBFC408B6DF77E2BE, VertexParallelism: 1, TaskAttemptID:attempt_1452786536144_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2330E9FD883747EA9628137A87F90C09, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B4EB7529D81A42E1A9E50ECF80EBE390, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:56,327 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536144_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:56,327 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:56,328 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:56,328 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:56,328 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:56,329 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786536144_0001_00_000001
2016-01-14 15:48:56,329 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:56,331 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786536144_0001_00_000001
2016-01-14 15:48:56,331 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:56,331 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:56,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:56,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786536144_0001_00_000001, launchTime=1452786536331
2016-01-14 15:48:56,333 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786536144_0001_00_000001
2016-01-14 15:48:56,333 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786536144_0001_1_00_000000_0] to container: [container_1452786536144_0001_00_000001]
2016-01-14 15:48:56,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:56,333 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786536144_0001_00_000001 given task: attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,333 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786536144_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,334 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786536144_0001_1_00_000000_0] started. Is using containerId: [container_1452786536144_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:56,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:56,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, taskAttemptId=attempt_1452786536144_0001_1_00_000000_0, startTime=1452786536333, containerId=container_1452786536144_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786536144_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:56,334 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536144_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:56,334 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536144_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:56 Starting to run new task attempt: attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,335 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,336 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,336 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:56,336 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:56,336 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:56,336 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:56,337 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:56,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A] comprehensivesort/(1/1) ...ive/false_null_false_true, VertexName: C3DBBE829BFD4AECBFC408B6DF77E2BE, VertexParallelism: 1, TaskAttemptID:attempt_1452786536144_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2330E9FD883747EA9628137A87F90C09, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B4EB7529D81A42E1A9E50ECF80EBE390, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:56,337 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:56,337 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:56,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,338 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786536144_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:56,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2330E9FD883747EA9628137A87F90C09, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:56,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:56,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2330E9FD883747EA9628137A87F90C09
2016-01-14 15:48:56,340 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2330E9FD883747EA9628137A87F90C09
2016-01-14 15:48:56,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B4EB7529D81A42E1A9E50ECF80EBE390, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:56,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:56,341 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:56,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B4EB7529D81A42E1A9E50ECF80EBE390
2016-01-14 15:48:56,342 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:56,342 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2330E9FD883747EA9628137A87F90C09
2016-01-14 15:48:56,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2330E9FD883747EA9628137A87F90C09
2016-01-14 15:48:56,344 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:56,344 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B4EB7529D81A42E1A9E50ECF80EBE390
2016-01-14 15:48:56,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B4EB7529D81A42E1A9E50ECF80EBE390
2016-01-14 15:48:56,347 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C3DBBE829BFD4AECBFC408B6DF77E2BE, ordinal: 0
2016-01-14 15:48:56,347 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3DBBE829BFD4AECBFC408B6DF77E2BE, mem on start (mb), free: 292, total: 474, max: 672
2016-01-14 15:48:56,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:56,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:56,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:56,348 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:56,348 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:56,348 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:56,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:56,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:56,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:56,349 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2330E9FD883747EA9628137A87F90C09, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:56,349 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B4EB7529D81A42E1A9E50ECF80EBE390, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:56,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:56,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2330E9FD883747EA9628137A87F90C09 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:56,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:56,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:56,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2330E9FD883747EA9628137A87F90C09
2016-01-14 15:48:56,351 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:56,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2330E9FD883747EA9628137A87F90C09
2016-01-14 15:48:56,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:56,352 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,352 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:56,353 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1b146a4a
2016-01-14 15:48:56,354 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1b146a4a
2016-01-14 15:48:56,354 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:56,354 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:56,355 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:56,355 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 2330E9FD883747EA9628137A87F90C09
2016-01-14 15:48:56,355 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: E09B5A452A5E4658B796458D73F71198
2016-01-14 15:48:56,356 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] E09B5A452A5E4658B796458D73F71198
2016-01-14 15:48:56,356 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@18923b79; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@287e9189; comparator=cascading.tuple.hadoop.util.TupleComparator@d84ce24; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:56,356 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:56,356 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:56,357 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:56,369 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:56,369 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:56,369 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:56,369 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:56,370 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:56,371 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:56,371 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:56,372 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C3DBBE829BFD4AECBFC408B6DF77E2BE, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:56,376 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,376 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C3DBBE829BFD4AECBFC408B6DF77E2BE, mem on close (mb), free: 190, total: 474, max: 672
2016-01-14 15:48:56,376 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:56,377 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:56,377 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:56,377 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:56,379 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:56 Completed running task attempt: attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,382 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786536144_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:56,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786536144_0001_1_00, eventCount=1
2016-01-14 15:48:56,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786536372, Process_Duration=4, Process_End_Time=1452786536376, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:56,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:56,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, taskAttemptId=attempt_1452786536144_0001_1_00_000000_0, startTime=1452786536333, finishTime=1452786536382, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536372, Process_Duration=4, Process_End_Time=1452786536376, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:56,383 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:56,383 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536144_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:56,383 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,383 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786536144_0001_1_00_000000_0
2016-01-14 15:48:56,383 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786536144_0001_00_000001
2016-01-14 15:48:56,383 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:56,384 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:56,384 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:TASK_FINISHED]: vertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, taskId=task_1452786536144_0001_1_00_000000, startTime=1452786536333, finishTime=1452786536383, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1452786536144_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536372, Process_Duration=4, Process_End_Time=1452786536376, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:56,384 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536144_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:56,384 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] : 1
2016-01-14 15:48:56,384 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:56,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:VERTEX_FINISHED]: vertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, vertexId=vertex_1452786536144_0001_1_00, initRequestedTime=1452786536303, initedTime=1452786536320, startRequestedTime=1452786536321, startedTime=1452786536321, finishTime=1452786536385, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536372, Process_Duration=4, Process_End_Time=1452786536376, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786536333, firstTasksToStart=[ task_1452786536144_0001_1_00_000000 ], lastTaskFinishTime=1452786536382, lastTasksToFinish=[ task_1452786536144_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786536144_0001_1_00_000000 ], longestDurationTasks=[ task_1452786536144_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:56,386 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:56,387 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:56,387 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390] attempt: attempt_1452786536144_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:56,387 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B4EB7529D81A42E1A9E50ECF80EBE390
2016-01-14 15:48:56,387 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786536144_0001_00_000001, stoppedTime=1452786536387, exitStatus=0
2016-01-14 15:48:56,387 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:56,387 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:56,388 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:56,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:TASK_STARTED]: vertexName=B4EB7529D81A42E1A9E50ECF80EBE390, taskId=task_1452786536144_0001_1_01_000000, scheduledTime=1452786536388, launchTime=1452786536388
2016-01-14 15:48:56,388 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536144_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:56,388 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786536144_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:56,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:56,388 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,389 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:56,389 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A] comprehensivesort/(1/1) ...ive/false_null_false_true, VertexName: B4EB7529D81A42E1A9E50ECF80EBE390, VertexParallelism: 1, TaskAttemptID:attempt_1452786536144_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B0B4E0F65D274D28BBC2A8964DBE12AE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:56,389 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536144_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:56,389 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:56,390 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786536144_0001_00_000002
2016-01-14 15:48:56,390 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:56,391 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786536144_0001_00_000002
2016-01-14 15:48:56,391 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:56,392 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:56,392 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:56,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786536144_0001_00_000002, launchTime=1452786536392
2016-01-14 15:48:56,393 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786536144_0001_00_000002
2016-01-14 15:48:56,394 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786536144_0001_1_01_000000_0] to container: [container_1452786536144_0001_00_000002]
2016-01-14 15:48:56,394 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:56,394 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786536144_0001_00_000002 given task: attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,394 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786536144_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,394 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786536144_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:56,394 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786536144_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:56,394 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786536144_0001_1_01_000000_0] started. Is using containerId: [container_1452786536144_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:56,394 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786536144_0001_00_000001, Exiting
2016-01-14 15:48:56,395 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786536144_0001_00_000001
2016-01-14 15:48:56,395 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786536144_0001_00_000001
2016-01-14 15:48:56 Starting to run new task attempt: attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,395 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,396 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B4EB7529D81A42E1A9E50ECF80EBE390, taskAttemptId=attempt_1452786536144_0001_1_01_000000_0, startTime=1452786536394, containerId=container_1452786536144_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786536144_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:56,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:56,396 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:56,396 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536144_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:56,396 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:56,396 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786536144_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:56,397 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786536144_0001_00_000001 completed successfully
2016-01-14 15:48:56,397 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:56,397 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:56,396 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536144_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:56,397 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:56,398 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:56,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A] comprehensivesort/(1/1) ...ive/false_null_false_true, VertexName: B4EB7529D81A42E1A9E50ECF80EBE390, VertexParallelism: 1, TaskAttemptID:attempt_1452786536144_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B0B4E0F65D274D28BBC2A8964DBE12AE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:56,398 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:56,399 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:56,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,400 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786536144_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:56,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C3DBBE829BFD4AECBFC408B6DF77E2BE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:56,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:56,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,402 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:56,402 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:56,402 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,406 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:56,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B0B4E0F65D274D28BBC2A8964DBE12AE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:56,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:56,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B0B4E0F65D274D28BBC2A8964DBE12AE
2016-01-14 15:48:56,408 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:56,409 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B0B4E0F65D274D28BBC2A8964DBE12AE
2016-01-14 15:48:56,413 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B4EB7529D81A42E1A9E50ECF80EBE390, ordinal: 1
2016-01-14 15:48:56,414 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B4EB7529D81A42E1A9E50ECF80EBE390, mem on start (mb), free: 183, total: 474, max: 672
2016-01-14 15:48:56,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:56,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:56,418 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:56,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B0B4E0F65D274D28BBC2A8964DBE12AE
2016-01-14 15:48:56,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:56,418 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:56,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:56,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:56,418 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:56,419 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:56,419 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:56,419 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C3DBBE829BFD4AECBFC408B6DF77E2BE, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:56,419 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B0B4E0F65D274D28BBC2A8964DBE12AE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:56,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:56,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C3DBBE829BFD4AECBFC408B6DF77E2BE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:56,420 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:56,420 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:56,420 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C3DBBE829BFD4AECBFC408B6DF77E2BE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:56,420 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:56,421 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:56,421 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:56,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:56,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:56,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:56,423 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,423 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786536144_0001_1_00_000000_0_10095, runDuration: 38627, ]
2016-01-14 15:48:56,425 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:56,426 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: E09B5A452A5E4658B796458D73F71198
2016-01-14 15:48:56,426 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786536144_0001_1_00_000000_0_10095] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786536426, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:56,426 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true"], id: B0B4E0F65D274D28BBC2A8964DBE12AE
2016-01-14 15:48:56,426 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,426 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true"]
2016-01-14 15:48:56,427 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:56,427 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] E09B5A452A5E4658B796458D73F71198, for 1 inputs
2016-01-14 15:48:56,427 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C3DBBE829BFD4AECBFC408B6DF77E2BE] #17 in 2ms
2016-01-14 15:48:56,427 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C3DBBE829BFD4AECBFC408B6DF77E2BE] #17, status:false, isInterrupted:false
2016-01-14 15:48:56,428 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:56,428 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:56,429 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:56,429 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:56,429 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:56,429 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,429 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B4EB7529D81A42E1A9E50ECF80EBE390, all 1 inputs ready in: 00:00:00.002
2016-01-14 15:48:56,429 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:56,430 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6f3ea4e3; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@79bb6c23
2016-01-14 15:48:56,434 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:56,434 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:56,434 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,434 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786536144_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:56,435 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,435 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B4EB7529D81A42E1A9E50ECF80EBE390, mem on close (mb), free: 180, total: 474, max: 672
2016-01-14 15:48:56,435 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C3DBBE829BFD4AECBFC408B6DF77E2BE
2016-01-14 15:48:56,435 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:56,435 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:56,435 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:56,436 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:56,436 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:56 Completed running task attempt: attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,436 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786536144_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:56,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786536429, Process_Duration=6, Process_End_Time=1452786536435, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:56,439 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:56,439 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,439 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786536144_0001_00_000002
2016-01-14 15:48:56,440 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:56,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B4EB7529D81A42E1A9E50ECF80EBE390, taskAttemptId=attempt_1452786536144_0001_1_01_000000_0, startTime=1452786536394, finishTime=1452786536439, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536429, Process_Duration=6, Process_End_Time=1452786536435, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:56,440 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536144_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:56,440 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:56,440 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786536144_0001_1_01_000000_0
2016-01-14 15:48:56,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:56,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:TASK_FINISHED]: vertexName=B4EB7529D81A42E1A9E50ECF80EBE390, taskId=task_1452786536144_0001_1_01_000000, startTime=1452786536394, finishTime=1452786536441, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1452786536144_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536429, Process_Duration=6, Process_End_Time=1452786536435, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:56,442 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536144_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:56,442 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390] : 1
2016-01-14 15:48:56,442 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:56,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:VERTEX_FINISHED]: vertexName=B4EB7529D81A42E1A9E50ECF80EBE390, vertexId=vertex_1452786536144_0001_1_01, initRequestedTime=1452786536304, initedTime=1452786536309, startRequestedTime=1452786536325, startedTime=1452786536325, finishTime=1452786536442, timeTaken=117, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536429, Process_Duration=6, Process_End_Time=1452786536435, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786536394, firstTasksToStart=[ task_1452786536144_0001_1_01_000000 ], lastTaskFinishTime=1452786536439, lastTasksToFinish=[ task_1452786536144_0001_1_01_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786536144_0001_1_01_000000 ], longestDurationTasks=[ task_1452786536144_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:56,443 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:56,443 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:56,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786536144_0001_00_000002, stoppedTime=1452786536443, exitStatus=0
2016-01-14 15:48:56,444 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:56,444 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:56,444 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786536144_0001_1
2016-01-14 15:48:56,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786536144_0001_1
2016-01-14 15:48:56,445 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B0B4E0F65D274D28BBC2A8964DBE12AE for vertex: vertex_1452786536144_0001_1_01 [B4EB7529D81A42E1A9E50ECF80EBE390]
2016-01-14 15:48:56,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:56,450 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786536144_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:56,450 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786536144_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:56,450 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786536144_0001_00_000002, Exiting
2016-01-14 15:48:56,450 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786536144_0001_00_000002
2016-01-14 15:48:56,450 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786536144_0001_00_000002
2016-01-14 15:48:56,450 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786536144_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:56,450 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786536144_0001_00_000002 completed successfully
2016-01-14 15:48:56,453 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786536144_0001_1_00 [C3DBBE829BFD4AECBFC408B6DF77E2BE]
2016-01-14 15:48:56,454 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786536144_0001_1, queueSize=0
2016-01-14 15:48:56,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:56,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536144_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786536144_0001_1, startTime=1452786536302, finishTime=1452786536453, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=1379, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=994050048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573072801, Process_Duration=10, Process_End_Time=2905573072811, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:56,455 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786536144_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:56,456 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786536144_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:56,456 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536144_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:56,456 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786536144_0001_00_000002
2016-01-14 15:48:56 Completed Dag: dag_1452786536144_0001_1
2016-01-14 15:48:56,457 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786536144_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:56,457 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786536144_0001_1_post
2016-01-14 15:48:56,457 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,458 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:56,461 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_null_false_true, applicationId=application_1452786536144_0001
2016-01-14 15:48:56,462 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:56,462 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:56,462 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:56,462 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:56,462 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:56,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:56,463 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:56,463 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:56,464 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:56,463 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:56,464 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:56,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:56,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:56,469 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A1275865E314E87941B697A914A5544/F7FAE054DF3543A69B95CE2BF76DE25A/.tez/application_1452786536144_0001
2016-01-14 15:48:56,469 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.347
2016-01-14 15:48:56,520 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:56,520 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:56,553 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 15:48:56,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-01-14 15:48:56,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:56,567 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:56,567 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:56,578 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:56,585 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true
2016-01-14 15:48:56,620 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:56,620 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:56,620 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true"]
2016-01-14 15:48:56,621 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:56,621 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:56,621 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:56,622 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_false_true
2016-01-14 15:48:56,622 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:56,630 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:56,635 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:56,636 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:56,636 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:56,645 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA/.tez/application_1452786536636_0001 doesn't exist and is created
2016-01-14 15:48:56,661 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:56,662 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:56,662 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA/.tez/application_1452786536636_0001
2016-01-14 15:48:56,665 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786536636_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:56,673 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:56,674 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:56,675 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:56,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:56,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA/.tez/application_1452786536636_0001/localmode-log-dir
2016-01-14 15:48:56,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA/.tez/application_1452786536636_0001/localmode-log-dir/history.txt.appattempt_1452786536636_0001_000000, maxErrors=10
2016-01-14 15:48:56,676 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:56,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786536636_0001_000000, appSubmitTime=1452786536665, launchTime=1452786536665
2016-01-14 15:48:56,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:56,682 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:56,683 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:56,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:56,685 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41087
2016-01-14 15:48:56,688 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:56,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786536636_0001_000000, startTime=1452786536691
2016-01-14 15:48:56,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:56,691 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:56,762 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:56,762 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:56,762 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_null_false_true, applicationId=application_1452786536636_0001, dagName=[0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA] comprehensivesort/(1/1) ...sive/true_null_false_true
2016-01-14 15:48:56,781 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA] comprehensivesort/(1/1) ...sive/true_null_false_true
2016-01-14 15:48:56,783 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786536636_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA/.tez/application_1452786536636_0001/localmode-log-dir/dag_1452786536636_0001_1.dot
2016-01-14 15:48:56,783 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA/.tez/application_1452786536636_0001/localmode-log-dir/dag_1452786536636_0001_1-tez-dag.pb.txt
2016-01-14 15:48:56,793 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786536636_0001_1
2016-01-14 15:48:56,793 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,793 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA] comprehensivesort/(1/1) ...sive/true_null_false_true
2016-01-14 15:48:56 Running Dag: dag_1452786536636_0001_1
2016-01-14 15:48:56,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:56,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786536636_0001_1, submitTime=1452786536782
2016-01-14 15:48:56,804 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E9FF6654BD804B0A9D4E42F905729E91
2016-01-14 15:48:56,805 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:56,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:56,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786536636_0001_1, initTime=1452786536804
2016-01-14 15:48:56,806 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786536636_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:56,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:DAG_STARTED]: dagID=dag_1452786536636_0001_1, startTime=1452786536806
2016-01-14 15:48:56,806 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:56,806 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786536636_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:56,806 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B9B4055436FC4C0EB2C42B25524DABE5 : {317336BBD13D45BE9682A21EF14E4213={InputName=317336BBD13D45BE9682A21EF14E4213}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:56,806 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 317336BBD13D45BE9682A21EF14E4213, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:56,806 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:56,807 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,807 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,807 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]. Starting root input initializers: 1
2016-01-14 15:48:56,807 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:56,808 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,808 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_null_false_true, applicationId=application_1452786536636_0001, dagName=[0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA] comprehensivesort/(1/1) ...sive/true_null_false_true
2016-01-14 15:48:56,809 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 317336BBD13D45BE9682A21EF14E4213 on vertex vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,811 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:56,811 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:56,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,812 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,812 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6A318DE13A8E461B9DC71E3DAC8F6F25, vertexId=vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:56,811 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 317336BBD13D45BE9682A21EF14E4213 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:56,812 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6A318DE13A8E461B9DC71E3DAC8F6F25, vertexId=vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,813 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:56,813 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:56,814 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6A318DE13A8E461B9DC71E3DAC8F6F25, vertexName=E9FF6654BD804B0A9D4E42F905729E91, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:56,815 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6A318DE13A8E461B9DC71E3DAC8F6F25, vertexId=vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,815 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:56,815 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 317336BBD13D45BE9682A21EF14E4213 on vertex vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:56,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E9FF6654BD804B0A9D4E42F905729E91, vertexId=vertex_1452786536636_0001_1_01, initRequestedTime=1452786536808, initedTime=1452786536816, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:56,816 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:56,817 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:56,817 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {317336BBD13D45BE9682A21EF14E4213=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:56,817 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] parallelism set to 1
2016-01-14 15:48:56,817 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:56,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786536636_0001_1_00, eventCount=1
2016-01-14 15:48:56,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B9B4055436FC4C0EB2C42B25524DABE5, vertexId=vertex_1452786536636_0001_1_00, initRequestedTime=1452786536806, initedTime=1452786536818, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:56,819 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] with distanceFromRoot: 0
2016-01-14 15:48:56,819 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:56,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:56,819 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,819 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786536636_0001_1_00, startRequestedTime=1452786536819, startedTime=1452786536819
2016-01-14 15:48:56,820 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:56,820 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:56,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:56,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:TASK_STARTED]: vertexName=B9B4055436FC4C0EB2C42B25524DABE5, taskId=task_1452786536636_0001_1_00_000000, scheduledTime=1452786536821, launchTime=1452786536821
2016-01-14 15:48:56,821 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536636_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:56,821 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786536636_0001_1_00 for vertex: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91] numStartedSources: 1 numSources: 1
2016-01-14 15:48:56,821 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91] with distanceFromRoot: 1
2016-01-14 15:48:56,822 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,822 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786536636_0001, with dag id: dag_1452786536636_0001_1
2016-01-14 15:48:56,823 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B9B4055436FC4C0EB2C42B25524DABE5 in vertex: E9FF6654BD804B0A9D4E42F905729E91
2016-01-14 15:48:56,823 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E9FF6654BD804B0A9D4E42F905729E91 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:56,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786536636_0001_1_01, startRequestedTime=1452786536822, startedTime=1452786536822
2016-01-14 15:48:56,823 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:56,823 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:56,824 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA] comprehensivesort/(1/1) ...sive/true_null_false_true, VertexName: B9B4055436FC4C0EB2C42B25524DABE5, VertexParallelism: 1, TaskAttemptID:attempt_1452786536636_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=317336BBD13D45BE9682A21EF14E4213, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9FF6654BD804B0A9D4E42F905729E91, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:56,824 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536636_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:56,824 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:56,825 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:56,825 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:56,825 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:56,826 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786536636_0001_00_000001
2016-01-14 15:48:56,826 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:56,829 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786536636_0001_00_000001
2016-01-14 15:48:56,829 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:56,830 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:56,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:56,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786536636_0001_00_000001, launchTime=1452786536830
2016-01-14 15:48:56,831 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786536636_0001_00_000001
2016-01-14 15:48:56,831 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786536636_0001_1_00_000000_0] to container: [container_1452786536636_0001_00_000001]
2016-01-14 15:48:56,832 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:56,832 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786536636_0001_1_00_000000_0] started. Is using containerId: [container_1452786536636_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:56,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B9B4055436FC4C0EB2C42B25524DABE5, taskAttemptId=attempt_1452786536636_0001_1_00_000000_0, startTime=1452786536832, containerId=container_1452786536636_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786536636_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:56,832 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536636_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:56,832 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:56,832 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536636_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:56,833 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786536636_0001_00_000001 given task: attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56,833 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786536636_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56 Starting to run new task attempt: attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56,834 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56,834 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,834 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:56,834 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:56,834 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:56,834 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:56,834 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:56,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA] comprehensivesort/(1/1) ...sive/true_null_false_true, VertexName: B9B4055436FC4C0EB2C42B25524DABE5, VertexParallelism: 1, TaskAttemptID:attempt_1452786536636_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=317336BBD13D45BE9682A21EF14E4213, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9FF6654BD804B0A9D4E42F905729E91, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:56,835 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:56,835 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:56,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,839 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786536636_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:56,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=317336BBD13D45BE9682A21EF14E4213, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:56,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:56,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 317336BBD13D45BE9682A21EF14E4213
2016-01-14 15:48:56,840 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 317336BBD13D45BE9682A21EF14E4213
2016-01-14 15:48:56,842 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:56,842 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 317336BBD13D45BE9682A21EF14E4213
2016-01-14 15:48:56,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 317336BBD13D45BE9682A21EF14E4213
2016-01-14 15:48:56,842 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:56,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E9FF6654BD804B0A9D4E42F905729E91, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:56,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:56,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E9FF6654BD804B0A9D4E42F905729E91
2016-01-14 15:48:56,847 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:56,848 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E9FF6654BD804B0A9D4E42F905729E91
2016-01-14 15:48:56,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E9FF6654BD804B0A9D4E42F905729E91
2016-01-14 15:48:56,850 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B9B4055436FC4C0EB2C42B25524DABE5, ordinal: 0
2016-01-14 15:48:56,850 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9B4055436FC4C0EB2C42B25524DABE5, mem on start (mb), free: 316, total: 474, max: 672
2016-01-14 15:48:56,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:56,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:56,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:56,850 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:56,851 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:56,851 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:56,851 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:56,851 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:56,852 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:56,852 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 317336BBD13D45BE9682A21EF14E4213, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:56,852 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E9FF6654BD804B0A9D4E42F905729E91, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:56,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:56,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 317336BBD13D45BE9682A21EF14E4213 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:56,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 317336BBD13D45BE9682A21EF14E4213
2016-01-14 15:48:56,853 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:56,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 317336BBD13D45BE9682A21EF14E4213
2016-01-14 15:48:56,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:56,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:56,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:56,854 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56,854 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:56,854 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7821d87d
2016-01-14 15:48:56,855 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7821d87d
2016-01-14 15:48:56,855 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:56,855 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:56,857 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:56,857 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 317336BBD13D45BE9682A21EF14E4213
2016-01-14 15:48:56,857 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 21B97A48DA2A4C199D551390B3760B04
2016-01-14 15:48:56,858 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 21B97A48DA2A4C199D551390B3760B04
2016-01-14 15:48:56,859 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@75dc95c7; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@64ed7844; comparator=cascading.tuple.hadoop.util.TupleComparator@2272ec20; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:56,859 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:56,859 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:56,859 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:56,872 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:56,872 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:56,872 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:56,873 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:56,873 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:56,873 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:56,873 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:56,874 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B9B4055436FC4C0EB2C42B25524DABE5, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:56,878 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56,878 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9B4055436FC4C0EB2C42B25524DABE5, mem on close (mb), free: 214, total: 474, max: 672
2016-01-14 15:48:56,878 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:56,878 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:56,879 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:56,879 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:56,881 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:56 Completed running task attempt: attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56,884 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786536636_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:56,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786536636_0001_1_00, eventCount=1
2016-01-14 15:48:56,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:56,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B9B4055436FC4C0EB2C42B25524DABE5, taskAttemptId=attempt_1452786536636_0001_1_00_000000_0, startTime=1452786536832, finishTime=1452786536885, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536874, Process_Duration=4, Process_End_Time=1452786536878, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:56,886 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536636_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:56,886 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786536636_0001_1_00_000000_0
2016-01-14 15:48:56,886 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:56,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:56,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:TASK_FINISHED]: vertexName=B9B4055436FC4C0EB2C42B25524DABE5, taskId=task_1452786536636_0001_1_00_000000, startTime=1452786536832, finishTime=1452786536886, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452786536636_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536874, Process_Duration=4, Process_End_Time=1452786536878, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:56,888 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536636_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:56,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786536874, Process_Duration=4, Process_End_Time=1452786536878, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:56,889 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] : 1
2016-01-14 15:48:56,889 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:56,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:VERTEX_FINISHED]: vertexName=B9B4055436FC4C0EB2C42B25524DABE5, vertexId=vertex_1452786536636_0001_1_00, initRequestedTime=1452786536806, initedTime=1452786536818, startRequestedTime=1452786536819, startedTime=1452786536819, finishTime=1452786536889, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536874, Process_Duration=4, Process_End_Time=1452786536878, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786536832, firstTasksToStart=[ task_1452786536636_0001_1_00_000000 ], lastTaskFinishTime=1452786536885, lastTasksToFinish=[ task_1452786536636_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786536636_0001_1_00_000000 ], longestDurationTasks=[ task_1452786536636_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:56,891 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:56,891 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:56,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786536636_0001_00_000001, stoppedTime=1452786536891, exitStatus=0
2016-01-14 15:48:56,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:56,891 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:56,891 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91] attempt: attempt_1452786536636_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:56,892 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E9FF6654BD804B0A9D4E42F905729E91
2016-01-14 15:48:56,892 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,892 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:56,892 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786536636_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:56,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:TASK_STARTED]: vertexName=E9FF6654BD804B0A9D4E42F905729E91, taskId=task_1452786536636_0001_1_01_000000, scheduledTime=1452786536892, launchTime=1452786536892
2016-01-14 15:48:56,893 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:56,893 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,893 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786536636_0001_00_000001
2016-01-14 15:48:56,893 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536636_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:56,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:56,893 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,893 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786536636_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:56,894 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786536636_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:56,894 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786536636_0001_00_000001, Exiting
2016-01-14 15:48:56,894 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786536636_0001_00_000001
2016-01-14 15:48:56,894 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786536636_0001_00_000001
2016-01-14 15:48:56,894 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:56,895 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA] comprehensivesort/(1/1) ...sive/true_null_false_true, VertexName: E9FF6654BD804B0A9D4E42F905729E91, VertexParallelism: 1, TaskAttemptID:attempt_1452786536636_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9B4055436FC4C0EB2C42B25524DABE5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6A318DE13A8E461B9DC71E3DAC8F6F25, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:56,895 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536636_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:56,895 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:56,896 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786536636_0001_00_000002
2016-01-14 15:48:56,896 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:56,896 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786536636_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:56,896 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786536636_0001_00_000001 completed successfully
2016-01-14 15:48:56,897 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:56,897 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786536636_0001_00_000002
2016-01-14 15:48:56,898 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:56,899 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:56,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:56,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786536636_0001_00_000002, launchTime=1452786536899
2016-01-14 15:48:56,900 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786536636_0001_00_000002
2016-01-14 15:48:56,900 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786536636_0001_1_01_000000_0] to container: [container_1452786536636_0001_00_000002]
2016-01-14 15:48:56,900 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:56,901 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786536636_0001_00_000002 given task: attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,901 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786536636_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786536636_0001_1_01_000000_0] started. Is using containerId: [container_1452786536636_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:56 Starting to run new task attempt: attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E9FF6654BD804B0A9D4E42F905729E91, taskAttemptId=attempt_1452786536636_0001_1_01_000000_0, startTime=1452786536901, containerId=container_1452786536636_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786536636_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:56,901 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:56,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536636_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:56,901 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,902 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,902 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536636_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:56,902 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:56,903 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:56,903 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:56,903 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:56,903 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:56,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA] comprehensivesort/(1/1) ...sive/true_null_false_true, VertexName: E9FF6654BD804B0A9D4E42F905729E91, VertexParallelism: 1, TaskAttemptID:attempt_1452786536636_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9B4055436FC4C0EB2C42B25524DABE5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6A318DE13A8E461B9DC71E3DAC8F6F25, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:56,904 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:56,904 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:56,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,905 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786536636_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:56,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6A318DE13A8E461B9DC71E3DAC8F6F25, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:56,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:56,907 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:56,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6A318DE13A8E461B9DC71E3DAC8F6F25
2016-01-14 15:48:56,909 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:56,910 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6A318DE13A8E461B9DC71E3DAC8F6F25
2016-01-14 15:48:56,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B9B4055436FC4C0EB2C42B25524DABE5, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:56,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:56,913 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E9FF6654BD804B0A9D4E42F905729E91, ordinal: 1
2016-01-14 15:48:56,913 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9FF6654BD804B0A9D4E42F905729E91, mem on start (mb), free: 209, total: 474, max: 672
2016-01-14 15:48:56,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:56,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:56,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,915 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:56,915 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:56,915 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:56,921 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:56,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6A318DE13A8E461B9DC71E3DAC8F6F25
2016-01-14 15:48:56,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:56,921 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:56,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:56,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:56,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:56,922 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:56,922 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:56,924 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6A318DE13A8E461B9DC71E3DAC8F6F25, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:56,924 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B9B4055436FC4C0EB2C42B25524DABE5, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:56,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:56,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B9B4055436FC4C0EB2C42B25524DABE5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:56,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:56,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:56,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,926 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:56,926 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:56,927 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B9B4055436FC4C0EB2C42B25524DABE5 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:56,927 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:56,927 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:56,927 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:56,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:56,928 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,930 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786536636_0001_1_00_000000_0_10101, runDuration: 38054, ]
2016-01-14 15:48:56,932 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:56,933 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 21B97A48DA2A4C199D551390B3760B04
2016-01-14 15:48:56,933 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true"], id: 6A318DE13A8E461B9DC71E3DAC8F6F25
2016-01-14 15:48:56,933 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true"]
2016-01-14 15:48:56,933 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 21B97A48DA2A4C199D551390B3760B04, for 1 inputs
2016-01-14 15:48:56,934 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786536636_0001_1_00_000000_0_10101] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786536934, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:56,934 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,934 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:56,935 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B9B4055436FC4C0EB2C42B25524DABE5] #18 in 2ms
2016-01-14 15:48:56,935 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B9B4055436FC4C0EB2C42B25524DABE5] #18, status:false, isInterrupted:false
2016-01-14 15:48:56,935 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:56,936 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:56,936 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:56,936 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:56,937 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:56,937 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,937 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E9FF6654BD804B0A9D4E42F905729E91, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:48:56,937 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:56,938 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@27b2533d; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2c76de27
2016-01-14 15:48:56,942 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:56,942 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:56,942 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,942 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786536636_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:56,943 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,943 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9FF6654BD804B0A9D4E42F905729E91, mem on close (mb), free: 205, total: 474, max: 672
2016-01-14 15:48:56,943 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B9B4055436FC4C0EB2C42B25524DABE5
2016-01-14 15:48:56,943 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:56,944 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:56,944 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:56,944 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:56,944 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:56 Completed running task attempt: attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,945 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786536636_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:56,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E9FF6654BD804B0A9D4E42F905729E91, taskAttemptId=attempt_1452786536636_0001_1_01_000000_0, startTime=1452786536901, finishTime=1452786536945, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536937, Process_Duration=6, Process_End_Time=1452786536943, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:56,946 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786536636_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:56,946 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:56,946 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786536636_0001_1_01_000000_0
2016-01-14 15:48:56,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:56,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:TASK_FINISHED]: vertexName=E9FF6654BD804B0A9D4E42F905729E91, taskId=task_1452786536636_0001_1_01_000000, startTime=1452786536901, finishTime=1452786536946, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452786536636_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536937, Process_Duration=6, Process_End_Time=1452786536943, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:56,947 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786536636_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:56,947 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91] : 1
2016-01-14 15:48:56,947 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:56,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:56,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:VERTEX_FINISHED]: vertexName=E9FF6654BD804B0A9D4E42F905729E91, vertexId=vertex_1452786536636_0001_1_01, initRequestedTime=1452786536808, initedTime=1452786536816, startRequestedTime=1452786536822, startedTime=1452786536822, finishTime=1452786536947, timeTaken=125, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786536937, Process_Duration=6, Process_End_Time=1452786536943, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786536901, firstTasksToStart=[ task_1452786536636_0001_1_01_000000 ], lastTaskFinishTime=1452786536945, lastTasksToFinish=[ task_1452786536636_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786536636_0001_1_01_000000 ], longestDurationTasks=[ task_1452786536636_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:56,948 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:56,949 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:56,949 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:56,949 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786536636_0001_1
2016-01-14 15:48:56,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786536636_0001_1
2016-01-14 15:48:56,950 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786536636_0001_1_00 [B9B4055436FC4C0EB2C42B25524DABE5]
2016-01-14 15:48:56,950 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6A318DE13A8E461B9DC71E3DAC8F6F25 for vertex: vertex_1452786536636_0001_1_01 [E9FF6654BD804B0A9D4E42F905729E91]
2016-01-14 15:48:56,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786536937, Process_Duration=6, Process_End_Time=1452786536943, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:56,953 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:56,953 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,953 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786536636_0001_00_000002
2016-01-14 15:48:56,958 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786536636_0001_1, queueSize=0
2016-01-14 15:48:56,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:56,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786536636_0001_1, startTime=1452786536806, finishTime=1452786536956, timeTaken=150, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=859, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=995098624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573073811, Process_Duration=10, Process_End_Time=2905573073821, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:56,959 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786536636_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:56,959 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786536636_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:56,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786536636_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786536636_0001_00_000002, stoppedTime=1452786536959, exitStatus=0
2016-01-14 15:48:56,959 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:56,960 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:56 Completed Dag: dag_1452786536636_0001_1
2016-01-14 15:48:56,961 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786536636_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:56,961 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786536636_0001_1_post
2016-01-14 15:48:56,962 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:56,962 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:56,962 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786536636_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:48:56,963 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:56,964 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786536636_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:56,964 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786536636_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:56,964 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786536636_0001_00_000002, Exiting
2016-01-14 15:48:56,964 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786536636_0001_00_000002
2016-01-14 15:48:56,965 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786536636_0001_00_000002
2016-01-14 15:48:56,965 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786536636_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:56,965 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786536636_0001_00_000002 completed successfully
2016-01-14 15:48:56,965 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786536636_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:56,966 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_null_false_true, applicationId=application_1452786536636_0001
2016-01-14 15:48:56,968 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:56,968 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:56,968 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:56,968 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:56,968 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:56,969 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:56,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:56,970 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:56,970 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:56,970 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:56,970 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:56,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:56,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:56,975 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0AE539A2D5F44A3BBB25B4318B5A0134/C5832D3C368843A1AAAB43DD69D1C7DA/.tez/application_1452786536636_0001
2016-01-14 15:48:56,975 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.355
2016-01-14 15:48:57,026 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:57,026 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:57,063 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 15:48:57,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 15:48:57,074 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:57,074 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:57,075 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:57,092 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:57,103 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true
2016-01-14 15:48:57,157 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:57,158 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:57,158 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true"]
2016-01-14 15:48:57,158 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:57,158 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:57,158 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:57,161 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...sive/false_null_true_true
2016-01-14 15:48:57,161 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:57,173 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:57,177 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:57,178 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:57,178 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:57,187 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB/.tez/application_1452786537178_0001 doesn't exist and is created
2016-01-14 15:48:57,203 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:57,203 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:57,204 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB/.tez/application_1452786537178_0001
2016-01-14 15:48:57,207 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786537178_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:57,219 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:57,220 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:57,222 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:57,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:57,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB/.tez/application_1452786537178_0001/localmode-log-dir
2016-01-14 15:48:57,223 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB/.tez/application_1452786537178_0001/localmode-log-dir/history.txt.appattempt_1452786537178_0001_000000, maxErrors=10
2016-01-14 15:48:57,223 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:57,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786537178_0001_000000, appSubmitTime=1452786537207, launchTime=1452786537207
2016-01-14 15:48:57,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:57,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:57,228 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:57,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:57,232 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:43730
2016-01-14 15:48:57,233 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:57,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786537178_0001_000000, startTime=1452786537235
2016-01-14 15:48:57,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:57,235 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:57,303 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:57,304 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:57,304 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/false_null_true_true, applicationId=application_1452786537178_0001, dagName=[00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB] comprehensivesort/(1/1) ...sive/false_null_true_true
2016-01-14 15:48:57,315 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB] comprehensivesort/(1/1) ...sive/false_null_true_true
2016-01-14 15:48:57,317 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786537178_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB/.tez/application_1452786537178_0001/localmode-log-dir/dag_1452786537178_0001_1.dot
2016-01-14 15:48:57,318 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB/.tez/application_1452786537178_0001/localmode-log-dir/dag_1452786537178_0001_1-tez-dag.pb.txt
2016-01-14 15:48:57,326 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786537178_0001_1
2016-01-14 15:48:57,326 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:57,326 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB] comprehensivesort/(1/1) ...sive/false_null_true_true
2016-01-14 15:48:57 Running Dag: dag_1452786537178_0001_1
2016-01-14 15:48:57,336 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:57,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786537178_0001_1, submitTime=1452786537316
2016-01-14 15:48:57,337 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F32C60AFAFEA424FAF8D427145F082C2
2016-01-14 15:48:57,337 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:57,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:57,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786537178_0001_1, initTime=1452786537337
2016-01-14 15:48:57,338 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786537178_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:57,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:DAG_STARTED]: dagID=dag_1452786537178_0001_1, startTime=1452786537338
2016-01-14 15:48:57,338 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:57,339 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786537178_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:57,339 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E4EF806198584F32BE1B26C2B9AFEA60 : {16516A52DA7D45768FF0DAAC02A34B1E={InputName=16516A52DA7D45768FF0DAAC02A34B1E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:57,339 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 16516A52DA7D45768FF0DAAC02A34B1E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:57,339 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,339 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,339 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,339 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]. Starting root input initializers: 1
2016-01-14 15:48:57,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:57,340 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:57,340 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,341 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/false_null_true_true, applicationId=application_1452786537178_0001, dagName=[00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB] comprehensivesort/(1/1) ...sive/false_null_true_true
2016-01-14 15:48:57,343 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:57,343 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,343 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,343 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,343 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2E6E155812694225AECD9AC2B7B95B57, vertexId=vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:57,343 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2E6E155812694225AECD9AC2B7B95B57, vertexId=vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,344 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 16516A52DA7D45768FF0DAAC02A34B1E on vertex vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,345 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2E6E155812694225AECD9AC2B7B95B57, vertexName=F32C60AFAFEA424FAF8D427145F082C2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:57,345 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2E6E155812694225AECD9AC2B7B95B57, vertexId=vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,345 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:57,346 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 16516A52DA7D45768FF0DAAC02A34B1E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:57,346 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:57,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:57,346 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:57,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F32C60AFAFEA424FAF8D427145F082C2, vertexId=vertex_1452786537178_0001_1_01, initRequestedTime=1452786537340, initedTime=1452786537346, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:57,347 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:57,347 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:57,347 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 16516A52DA7D45768FF0DAAC02A34B1E on vertex vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,348 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:57,348 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {16516A52DA7D45768FF0DAAC02A34B1E=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:57,348 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] parallelism set to 1
2016-01-14 15:48:57,348 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:57,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786537178_0001_1_00, eventCount=1
2016-01-14 15:48:57,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:57,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4EF806198584F32BE1B26C2B9AFEA60, vertexId=vertex_1452786537178_0001_1_00, initRequestedTime=1452786537339, initedTime=1452786537348, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:57,349 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] with distanceFromRoot: 0
2016-01-14 15:48:57,349 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:57,349 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,349 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,349 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:57,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786537178_0001_1_00, startRequestedTime=1452786537349, startedTime=1452786537349
2016-01-14 15:48:57,350 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:57,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:57,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:TASK_STARTED]: vertexName=E4EF806198584F32BE1B26C2B9AFEA60, taskId=task_1452786537178_0001_1_00_000000, scheduledTime=1452786537350, launchTime=1452786537350
2016-01-14 15:48:57,350 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537178_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:57,350 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786537178_0001_1_00 for vertex: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2] numStartedSources: 1 numSources: 1
2016-01-14 15:48:57,350 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2] with distanceFromRoot: 1
2016-01-14 15:48:57,351 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,351 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E4EF806198584F32BE1B26C2B9AFEA60 in vertex: F32C60AFAFEA424FAF8D427145F082C2
2016-01-14 15:48:57,351 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F32C60AFAFEA424FAF8D427145F082C2 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:57,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:57,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786537178_0001_1_01, startRequestedTime=1452786537351, startedTime=1452786537351
2016-01-14 15:48:57,352 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:57,352 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB] comprehensivesort/(1/1) ...sive/false_null_true_true, VertexName: E4EF806198584F32BE1B26C2B9AFEA60, VertexParallelism: 1, TaskAttemptID:attempt_1452786537178_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=16516A52DA7D45768FF0DAAC02A34B1E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F32C60AFAFEA424FAF8D427145F082C2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:57,352 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537178_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:57,352 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:57,353 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:57,353 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:57,353 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:57,354 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786537178_0001_00_000001
2016-01-14 15:48:57,354 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:57,355 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786537178_0001_00_000001
2016-01-14 15:48:57,355 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:57,356 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:57,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:57,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786537178_0001_00_000001, launchTime=1452786537356
2016-01-14 15:48:57,357 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786537178_0001_00_000001
2016-01-14 15:48:57,357 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786537178_0001_1_00_000000_0] to container: [container_1452786537178_0001_00_000001]
2016-01-14 15:48:57,357 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:57,358 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786537178_0001_00_000001 given task: attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,358 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786537178_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,358 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786537178_0001_1_00_000000_0] started. Is using containerId: [container_1452786537178_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:57 Starting to run new task attempt: attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,358 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,358 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:57,358 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:57,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4EF806198584F32BE1B26C2B9AFEA60, taskAttemptId=attempt_1452786537178_0001_1_00_000000_0, startTime=1452786537358, containerId=container_1452786537178_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786537178_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:57,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:57,359 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786537178_0001, with dag id: dag_1452786537178_0001_1
2016-01-14 15:48:57,358 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:57,359 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:57,360 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:57,360 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:57,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB] comprehensivesort/(1/1) ...sive/false_null_true_true, VertexName: E4EF806198584F32BE1B26C2B9AFEA60, VertexParallelism: 1, TaskAttemptID:attempt_1452786537178_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=16516A52DA7D45768FF0DAAC02A34B1E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F32C60AFAFEA424FAF8D427145F082C2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:57,360 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:57,359 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537178_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:57,361 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537178_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:57,361 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:57,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=16516A52DA7D45768FF0DAAC02A34B1E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:57,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:57,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F32C60AFAFEA424FAF8D427145F082C2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:57,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:57,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F32C60AFAFEA424FAF8D427145F082C2
2016-01-14 15:48:57,367 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786537178_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:57,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 16516A52DA7D45768FF0DAAC02A34B1E
2016-01-14 15:48:57,368 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:57,368 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 16516A52DA7D45768FF0DAAC02A34B1E
2016-01-14 15:48:57,369 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:57,369 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F32C60AFAFEA424FAF8D427145F082C2
2016-01-14 15:48:57,371 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:57,371 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 16516A52DA7D45768FF0DAAC02A34B1E
2016-01-14 15:48:57,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 16516A52DA7D45768FF0DAAC02A34B1E
2016-01-14 15:48:57,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F32C60AFAFEA424FAF8D427145F082C2
2016-01-14 15:48:57,375 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4EF806198584F32BE1B26C2B9AFEA60, ordinal: 0
2016-01-14 15:48:57,375 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4EF806198584F32BE1B26C2B9AFEA60, mem on start (mb), free: 118, total: 474, max: 672
2016-01-14 15:48:57,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:57,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:57,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:57,376 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:57,376 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:57,376 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:57,376 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:57,376 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:57,376 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:57,376 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 16516A52DA7D45768FF0DAAC02A34B1E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:57,377 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F32C60AFAFEA424FAF8D427145F082C2, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:57,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:57,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 16516A52DA7D45768FF0DAAC02A34B1E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:57,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:57,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:57,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 16516A52DA7D45768FF0DAAC02A34B1E
2016-01-14 15:48:57,378 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:57,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 16516A52DA7D45768FF0DAAC02A34B1E
2016-01-14 15:48:57,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:57,379 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,379 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:57,379 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@250bac37
2016-01-14 15:48:57,380 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@250bac37
2016-01-14 15:48:57,380 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:57,380 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:57,382 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:57,382 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 16516A52DA7D45768FF0DAAC02A34B1E
2016-01-14 15:48:57,382 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 82093987673F4F6CA5E8776A455AD49A
2016-01-14 15:48:57,383 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 82093987673F4F6CA5E8776A455AD49A
2016-01-14 15:48:57,383 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@620b64f0; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5a0ffad9; comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@4f1deac0; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:57,383 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:57,384 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:57,384 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:57,406 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:57,406 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:57,407 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:57,407 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:57,407 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:57,409 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:57,409 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:57,411 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4EF806198584F32BE1B26C2B9AFEA60, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:57,414 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,415 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4EF806198584F32BE1B26C2B9AFEA60, mem on close (mb), free: 231, total: 474, max: 672
2016-01-14 15:48:57,415 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:57,415 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:57,415 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:57,415 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:57,417 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:57 Completed running task attempt: attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,420 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786537178_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:57,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786537178_0001_1_00, eventCount=1
2016-01-14 15:48:57,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1145, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786537411, Process_Duration=3, Process_End_Time=1452786537414, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:57,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4EF806198584F32BE1B26C2B9AFEA60, taskAttemptId=attempt_1452786537178_0001_1_00_000000_0, startTime=1452786537358, finishTime=1452786537421, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1145, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786537411, Process_Duration=3, Process_End_Time=1452786537414, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:57,422 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537178_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:57,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:57,422 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786537178_0001_1_00_000000_0
2016-01-14 15:48:57,422 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:57,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:TASK_FINISHED]: vertexName=E4EF806198584F32BE1B26C2B9AFEA60, taskId=task_1452786537178_0001_1_00_000000, startTime=1452786537358, finishTime=1452786537422, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1452786537178_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1145, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786537411, Process_Duration=3, Process_End_Time=1452786537414, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:57,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:57,423 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537178_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:57,423 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] : 1
2016-01-14 15:48:57,423 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,424 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:57,424 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:57,424 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786537178_0001_00_000001
2016-01-14 15:48:57,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:VERTEX_FINISHED]: vertexName=E4EF806198584F32BE1B26C2B9AFEA60, vertexId=vertex_1452786537178_0001_1_00, initRequestedTime=1452786537339, initedTime=1452786537348, startRequestedTime=1452786537349, startedTime=1452786537349, finishTime=1452786537423, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1145, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786537411, Process_Duration=3, Process_End_Time=1452786537414, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786537358, firstTasksToStart=[ task_1452786537178_0001_1_00_000000 ], lastTaskFinishTime=1452786537421, lastTasksToFinish=[ task_1452786537178_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786537178_0001_1_00_000000 ], longestDurationTasks=[ task_1452786537178_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:57,424 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:57,424 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:57,425 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:57,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786537178_0001_00_000001, stoppedTime=1452786537425, exitStatus=0
2016-01-14 15:48:57,425 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:57,425 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2] attempt: attempt_1452786537178_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:57,425 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F32C60AFAFEA424FAF8D427145F082C2
2016-01-14 15:48:57,425 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,426 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:57,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:57,426 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786537178_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:57,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:TASK_STARTED]: vertexName=F32C60AFAFEA424FAF8D427145F082C2, taskId=task_1452786537178_0001_1_01_000000, scheduledTime=1452786537426, launchTime=1452786537426
2016-01-14 15:48:57,426 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537178_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:57,426 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:57,427 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,427 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB] comprehensivesort/(1/1) ...sive/false_null_true_true, VertexName: F32C60AFAFEA424FAF8D427145F082C2, VertexParallelism: 1, TaskAttemptID:attempt_1452786537178_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4EF806198584F32BE1B26C2B9AFEA60, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2E6E155812694225AECD9AC2B7B95B57, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:57,427 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537178_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:57,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:57,427 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:57,428 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786537178_0001_00_000002
2016-01-14 15:48:57,428 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:57,429 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786537178_0001_00_000002
2016-01-14 15:48:57,429 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:57,430 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:57,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:57,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786537178_0001_00_000002, launchTime=1452786537430
2016-01-14 15:48:57,431 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786537178_0001_00_000002
2016-01-14 15:48:57,431 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786537178_0001_1_01_000000_0] to container: [container_1452786537178_0001_00_000002]
2016-01-14 15:48:57,431 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:57,432 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786537178_0001_00_000002 given task: attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786537178_0001_1_01_000000_0] started. Is using containerId: [container_1452786537178_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:57,432 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786537178_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F32C60AFAFEA424FAF8D427145F082C2, taskAttemptId=attempt_1452786537178_0001_1_01_000000_0, startTime=1452786537432, containerId=container_1452786537178_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786537178_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:57,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537178_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:57 Starting to run new task attempt: attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,432 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537178_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:57,432 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,433 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:57,434 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:57,434 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:57,433 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:57,434 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:57,434 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786537178_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:57,435 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786537178_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:57,435 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786537178_0001_00_000001, Exiting
2016-01-14 15:48:57,435 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786537178_0001_00_000001
2016-01-14 15:48:57,435 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786537178_0001_00_000001
2016-01-14 15:48:57,435 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:57,436 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:57,436 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786537178_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:57,436 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786537178_0001_00_000001 completed successfully
2016-01-14 15:48:57,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB] comprehensivesort/(1/1) ...sive/false_null_true_true, VertexName: F32C60AFAFEA424FAF8D427145F082C2, VertexParallelism: 1, TaskAttemptID:attempt_1452786537178_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4EF806198584F32BE1B26C2B9AFEA60, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2E6E155812694225AECD9AC2B7B95B57, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:57,436 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:57,437 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:57,437 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:57,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E4EF806198584F32BE1B26C2B9AFEA60, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:57,439 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786537178_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:57,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:57,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2E6E155812694225AECD9AC2B7B95B57, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:57,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:57,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2E6E155812694225AECD9AC2B7B95B57
2016-01-14 15:48:57,440 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:57,440 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2E6E155812694225AECD9AC2B7B95B57
2016-01-14 15:48:57,442 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:57,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,445 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:57,445 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:57,445 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,449 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F32C60AFAFEA424FAF8D427145F082C2, ordinal: 1
2016-01-14 15:48:57,449 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F32C60AFAFEA424FAF8D427145F082C2, mem on start (mb), free: 225, total: 474, max: 672
2016-01-14 15:48:57,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:57,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:57,450 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:57,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2E6E155812694225AECD9AC2B7B95B57
2016-01-14 15:48:57,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:57,451 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:57,451 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:57,451 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:57,451 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:57,452 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:57,452 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:57,452 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2E6E155812694225AECD9AC2B7B95B57, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:57,452 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E4EF806198584F32BE1B26C2B9AFEA60, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:57,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:57,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E4EF806198584F32BE1B26C2B9AFEA60 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:57,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:57,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:57,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,454 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:57,454 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:57,455 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E4EF806198584F32BE1B26C2B9AFEA60 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:57,455 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:57,455 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:57,457 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:57,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:57,459 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,460 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786537178_0001_1_00_000000_0_10107, runDuration: 51738, ]
2016-01-14 15:48:57,461 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786537178_0001_1_00_000000_0_10107] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786537461, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:57,461 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,461 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:57,462 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E4EF806198584F32BE1B26C2B9AFEA60] #19 in 2ms
2016-01-14 15:48:57,462 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E4EF806198584F32BE1B26C2B9AFEA60] #19, status:false, isInterrupted:false
2016-01-14 15:48:57,462 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:57,463 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:57,463 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:57,463 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:57,463 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:57,463 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:57,463 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,464 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:57,464 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 82093987673F4F6CA5E8776A455AD49A
2016-01-14 15:48:57,464 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true"], id: 2E6E155812694225AECD9AC2B7B95B57
2016-01-14 15:48:57,465 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true"]
2016-01-14 15:48:57,465 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 82093987673F4F6CA5E8776A455AD49A, for 1 inputs
2016-01-14 15:48:57,465 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F32C60AFAFEA424FAF8D427145F082C2, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:57,466 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1481484b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@33b13e71
2016-01-14 15:48:57,469 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:57,469 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:57,469 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,470 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786537178_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:57,470 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,470 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F32C60AFAFEA424FAF8D427145F082C2, mem on close (mb), free: 222, total: 474, max: 672
2016-01-14 15:48:57,470 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E4EF806198584F32BE1B26C2B9AFEA60
2016-01-14 15:48:57,471 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:57,471 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:57,471 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:57,471 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:57,471 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:57 Completed running task attempt: attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,472 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786537178_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:57,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786537465, Process_Duration=5, Process_End_Time=1452786537470, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:57,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F32C60AFAFEA424FAF8D427145F082C2, taskAttemptId=attempt_1452786537178_0001_1_01_000000_0, startTime=1452786537432, finishTime=1452786537473, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786537465, Process_Duration=5, Process_End_Time=1452786537470, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:57,473 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:57,473 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537178_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:57,473 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:57,473 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:57,473 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:57,473 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786537178_0001_1_01_000000_0
2016-01-14 15:48:57,474 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786537178_0001_00_000002
2016-01-14 15:48:57,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:57,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:TASK_FINISHED]: vertexName=F32C60AFAFEA424FAF8D427145F082C2, taskId=task_1452786537178_0001_1_01_000000, startTime=1452786537432, finishTime=1452786537474, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1452786537178_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786537465, Process_Duration=5, Process_End_Time=1452786537470, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:57,475 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537178_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:57,475 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2] : 1
2016-01-14 15:48:57,475 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,476 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:57,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:VERTEX_FINISHED]: vertexName=F32C60AFAFEA424FAF8D427145F082C2, vertexId=vertex_1452786537178_0001_1_01, initRequestedTime=1452786537340, initedTime=1452786537346, startRequestedTime=1452786537351, startedTime=1452786537351, finishTime=1452786537475, timeTaken=124, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497549312, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786537465, Process_Duration=5, Process_End_Time=1452786537470, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786537432, firstTasksToStart=[ task_1452786537178_0001_1_01_000000 ], lastTaskFinishTime=1452786537473, lastTasksToFinish=[ task_1452786537178_0001_1_01_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786537178_0001_1_01_000000 ], longestDurationTasks=[ task_1452786537178_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:57,476 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:57,476 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:57,477 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:57,477 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786537178_0001_1
2016-01-14 15:48:57,477 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786537178_0001_1
2016-01-14 15:48:57,477 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2E6E155812694225AECD9AC2B7B95B57 for vertex: vertex_1452786537178_0001_1_01 [F32C60AFAFEA424FAF8D427145F082C2]
2016-01-14 15:48:57,486 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786537178_0001_1_00 [E4EF806198584F32BE1B26C2B9AFEA60]
2016-01-14 15:48:57,494 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786537178_0001_1, queueSize=0
2016-01-14 15:48:57,494 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:57,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786537178_0001_1, startTime=1452786537338, finishTime=1452786537486, timeTaken=148, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=1899, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=995098624, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573074876, Process_Duration=8, Process_End_Time=2905573074884, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:57,495 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786537178_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:57,495 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786537178_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:57,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537178_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786537178_0001_00_000002, stoppedTime=1452786537495, exitStatus=0
2016-01-14 15:48:57,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:57,497 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:57 Completed Dag: dag_1452786537178_0001_1
2016-01-14 15:48:57,497 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786537178_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:57,498 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786537178_0001_1_post
2016-01-14 15:48:57,498 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:57,498 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:57,499 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786537178_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:48:57,499 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:57,501 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sive/false_null_true_true, applicationId=application_1452786537178_0001
2016-01-14 15:48:57,502 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:57,502 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:57,502 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:57,503 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:57,503 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:57,503 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:57,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:57,504 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:57,504 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:57,504 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:57,505 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:57,505 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786537178_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:57,505 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786537178_0001_00_000002: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:57,505 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786537178_0001_00_000002, Exiting
2016-01-14 15:48:57,505 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786537178_0001_00_000002
2016-01-14 15:48:57,505 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786537178_0001_00_000002
2016-01-14 15:48:57,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786537178_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:57,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786537178_0001_00_000002 completed successfully
2016-01-14 15:48:57,506 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537178_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:57,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:57,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:57,512 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/00BF33C5BB3242149AC35B6A9B24E7D3/FB933445FEA94AD081881C16BFD447BB/.tez/application_1452786537178_0001
2016-01-14 15:48:57,513 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.356
2016-01-14 15:48:57,591 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:57,592 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:57,638 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 15:48:57,652 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 15:48:57,653 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:57,653 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:57,653 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:57,668 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:57,673 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true
2016-01-14 15:48:57,707 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:57,708 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:57,708 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true"]
2016-01-14 15:48:57,709 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:57,709 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:57,709 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:57,710 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...nsive/true_null_true_true
2016-01-14 15:48:57,710 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:57,719 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:57,723 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:57,724 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:57,724 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:57,733 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE/.tez/application_1452786537723_0001 doesn't exist and is created
2016-01-14 15:48:57,750 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:57,750 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:57,750 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE/.tez/application_1452786537723_0001
2016-01-14 15:48:57,753 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786537723_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:57,760 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:57,761 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:57,762 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:57,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:57,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE/.tez/application_1452786537723_0001/localmode-log-dir
2016-01-14 15:48:57,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE/.tez/application_1452786537723_0001/localmode-log-dir/history.txt.appattempt_1452786537723_0001_000000, maxErrors=10
2016-01-14 15:48:57,763 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:57,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786537723_0001_000000, appSubmitTime=1452786537752, launchTime=1452786537752
2016-01-14 15:48:57,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:57,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:57,769 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:57,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:57,773 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39146
2016-01-14 15:48:57,774 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:57,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786537723_0001_000000, startTime=1452786537776
2016-01-14 15:48:57,776 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:57,776 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:57,850 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:57,850 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:57,850 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...nsive/true_null_true_true, applicationId=application_1452786537723_0001, dagName=[BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE] comprehensivesort/(1/1) ...nsive/true_null_true_true
2016-01-14 15:48:57,869 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE] comprehensivesort/(1/1) ...nsive/true_null_true_true
2016-01-14 15:48:57,871 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786537723_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE/.tez/application_1452786537723_0001/localmode-log-dir/dag_1452786537723_0001_1.dot
2016-01-14 15:48:57,871 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE/.tez/application_1452786537723_0001/localmode-log-dir/dag_1452786537723_0001_1-tez-dag.pb.txt
2016-01-14 15:48:57,882 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786537723_0001_1
2016-01-14 15:48:57,883 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:57,883 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE] comprehensivesort/(1/1) ...nsive/true_null_true_true
2016-01-14 15:48:57 Running Dag: dag_1452786537723_0001_1
2016-01-14 15:48:57,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786537723_0001_1, submitTime=1452786537869
2016-01-14 15:48:57,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:57,898 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C58D06DE37714D9E9365AF34C305F85B
2016-01-14 15:48:57,898 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:57,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:57,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786537723_0001_1, initTime=1452786537897
2016-01-14 15:48:57,900 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786537723_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:57,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:DAG_STARTED]: dagID=dag_1452786537723_0001_1, startTime=1452786537900
2016-01-14 15:48:57,900 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:57,901 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786537723_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:57,901 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C43ABAFA369C41CCAA0DF9C8DF5E27C9 : {3E69E43EE8744324B90AF3E19B3E3458={InputName=3E69E43EE8744324B90AF3E19B3E3458}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:57,901 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3E69E43EE8744324B90AF3E19B3E3458, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:57,901 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:57,901 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:57,901 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:57,902 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]. Starting root input initializers: 1
2016-01-14 15:48:57,902 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:57,902 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:57,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:57,905 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:57,905 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:57,905 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:57,905 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:57,905 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8AD4FA6AB68E47BA9B5CA68A0C3035FC, vertexId=vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:57,905 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8AD4FA6AB68E47BA9B5CA68A0C3035FC, vertexId=vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:57,907 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8AD4FA6AB68E47BA9B5CA68A0C3035FC, vertexName=C58D06DE37714D9E9365AF34C305F85B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:57,908 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...nsive/true_null_true_true, applicationId=application_1452786537723_0001, dagName=[BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE] comprehensivesort/(1/1) ...nsive/true_null_true_true
2016-01-14 15:48:57,907 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3E69E43EE8744324B90AF3E19B3E3458 on vertex vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:57,908 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8AD4FA6AB68E47BA9B5CA68A0C3035FC, vertexId=vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:57,914 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:57,915 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3E69E43EE8744324B90AF3E19B3E3458 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:57,915 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:57,915 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:57,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C58D06DE37714D9E9365AF34C305F85B, vertexId=vertex_1452786537723_0001_1_01, initRequestedTime=1452786537902, initedTime=1452786537914, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:57,915 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:57,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:57,917 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:57,917 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3E69E43EE8744324B90AF3E19B3E3458 on vertex vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:57,917 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:57,918 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3E69E43EE8744324B90AF3E19B3E3458=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:57,918 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] parallelism set to 1
2016-01-14 15:48:57,918 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:57,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786537723_0001_1_00, eventCount=1
2016-01-14 15:48:57,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, vertexId=vertex_1452786537723_0001_1_00, initRequestedTime=1452786537901, initedTime=1452786537918, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:57,918 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] with distanceFromRoot: 0
2016-01-14 15:48:57,918 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:57,919 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:57,919 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:57,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786537723_0001_1_00, startRequestedTime=1452786537919, startedTime=1452786537919
2016-01-14 15:48:57,919 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:57,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:TASK_STARTED]: vertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, taskId=task_1452786537723_0001_1_00_000000, scheduledTime=1452786537919, launchTime=1452786537919
2016-01-14 15:48:57,919 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537723_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:57,920 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786537723_0001_1_00 for vertex: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B] numStartedSources: 1 numSources: 1
2016-01-14 15:48:57,920 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B] with distanceFromRoot: 1
2016-01-14 15:48:57,920 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:57,920 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C43ABAFA369C41CCAA0DF9C8DF5E27C9 in vertex: C58D06DE37714D9E9365AF34C305F85B
2016-01-14 15:48:57,920 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C58D06DE37714D9E9365AF34C305F85B with 1 source tasks and 1 pending tasks
2016-01-14 15:48:57,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786537723_0001_1_01, startRequestedTime=1452786537920, startedTime=1452786537920
2016-01-14 15:48:57,921 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:57,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE] comprehensivesort/(1/1) ...nsive/true_null_true_true, VertexName: C43ABAFA369C41CCAA0DF9C8DF5E27C9, VertexParallelism: 1, TaskAttemptID:attempt_1452786537723_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3E69E43EE8744324B90AF3E19B3E3458, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C58D06DE37714D9E9365AF34C305F85B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:57,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537723_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:57,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:57,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:57,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:57,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:57,923 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:57,923 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:57,924 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:57,924 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:57,924 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786537723_0001_00_000001
2016-01-14 15:48:57,924 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:57,926 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786537723_0001_00_000001
2016-01-14 15:48:57,926 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:57,927 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786537723_0001, with dag id: dag_1452786537723_0001_1
2016-01-14 15:48:57,933 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:57,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:57,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786537723_0001_00_000001, launchTime=1452786537933
2016-01-14 15:48:57,934 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786537723_0001_00_000001
2016-01-14 15:48:57,934 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786537723_0001_1_00_000000_0] to container: [container_1452786537723_0001_00_000001]
2016-01-14 15:48:57,935 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:57,935 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786537723_0001_1_00_000000_0] started. Is using containerId: [container_1452786537723_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:57,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, taskAttemptId=attempt_1452786537723_0001_1_00_000000_0, startTime=1452786537935, containerId=container_1452786537723_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786537723_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:57,936 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537723_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:57,936 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537723_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:57,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:57,936 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786537723_0001_00_000001 given task: attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:57,936 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786537723_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:57 Starting to run new task attempt: attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:57,937 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:57,937 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:57,937 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:57,937 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:57,938 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:57,938 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:57,938 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:57,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE] comprehensivesort/(1/1) ...nsive/true_null_true_true, VertexName: C43ABAFA369C41CCAA0DF9C8DF5E27C9, VertexParallelism: 1, TaskAttemptID:attempt_1452786537723_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3E69E43EE8744324B90AF3E19B3E3458, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C58D06DE37714D9E9365AF34C305F85B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:57,938 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:57,941 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:57,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:57,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3E69E43EE8744324B90AF3E19B3E3458, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:57,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:57,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3E69E43EE8744324B90AF3E19B3E3458
2016-01-14 15:48:57,943 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3E69E43EE8744324B90AF3E19B3E3458
2016-01-14 15:48:57,945 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:57,945 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3E69E43EE8744324B90AF3E19B3E3458
2016-01-14 15:48:57,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3E69E43EE8744324B90AF3E19B3E3458
2016-01-14 15:48:57,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,948 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:57,950 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786537723_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:57,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C58D06DE37714D9E9365AF34C305F85B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:57,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:57,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C58D06DE37714D9E9365AF34C305F85B
2016-01-14 15:48:57,956 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:57,956 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C58D06DE37714D9E9365AF34C305F85B
2016-01-14 15:48:57,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C58D06DE37714D9E9365AF34C305F85B
2016-01-14 15:48:57,958 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C43ABAFA369C41CCAA0DF9C8DF5E27C9, ordinal: 0
2016-01-14 15:48:57,958 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C43ABAFA369C41CCAA0DF9C8DF5E27C9, mem on start (mb), free: 137, total: 474, max: 672
2016-01-14 15:48:57,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:57,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:57,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:57,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:57,959 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:57,959 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:57,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:57,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:57,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:57,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:57,961 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3E69E43EE8744324B90AF3E19B3E3458, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:57,961 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C58D06DE37714D9E9365AF34C305F85B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:57,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:57,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3E69E43EE8744324B90AF3E19B3E3458
2016-01-14 15:48:57,961 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:57,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3E69E43EE8744324B90AF3E19B3E3458
2016-01-14 15:48:57,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3E69E43EE8744324B90AF3E19B3E3458 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:57,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:57,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:57,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:57,964 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:57,968 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:57,968 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 3E69E43EE8744324B90AF3E19B3E3458
2016-01-14 15:48:57,968 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: FF259DC655D646628D669E8206E61B56
2016-01-14 15:48:57,969 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] FF259DC655D646628D669E8206E61B56
2016-01-14 15:48:57,970 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:57,971 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7ee4848f; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4cbb095e; comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@b6421ad; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:57,971 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:57,971 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:57,971 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5adeeab2
2016-01-14 15:48:57,974 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:57,975 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5adeeab2
2016-01-14 15:48:57,975 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:57,985 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:57,999 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:57,999 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:57,999 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:58,000 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:58,000 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:58,000 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:58,001 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:58,001 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C43ABAFA369C41CCAA0DF9C8DF5E27C9, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:58,006 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:58,006 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C43ABAFA369C41CCAA0DF9C8DF5E27C9, mem on close (mb), free: 219, total: 464, max: 672
2016-01-14 15:48:58,007 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:58,007 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:58,007 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:58,007 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:58,011 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:58 Completed running task attempt: attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:58,015 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786537723_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:58,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786537723_0001_1_00, eventCount=1
2016-01-14 15:48:58,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, taskAttemptId=attempt_1452786537723_0001_1_00_000000_0, startTime=1452786537935, finishTime=1452786538016, timeTaken=81, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=487063552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538001, Process_Duration=5, Process_End_Time=1452786538006, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:58,016 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537723_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:58,017 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:58,017 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786537723_0001_1_00_000000_0
2016-01-14 15:48:58,017 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:58,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:TASK_FINISHED]: vertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, taskId=task_1452786537723_0001_1_00_000000, startTime=1452786537935, finishTime=1452786538017, timeTaken=82, status=SUCCEEDED, successfulAttemptID=attempt_1452786537723_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=487063552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538001, Process_Duration=5, Process_End_Time=1452786538006, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:58,018 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537723_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:58,018 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] : 1
2016-01-14 15:48:58,018 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:58,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:VERTEX_FINISHED]: vertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, vertexId=vertex_1452786537723_0001_1_00, initRequestedTime=1452786537901, initedTime=1452786537918, startRequestedTime=1452786537919, startedTime=1452786537919, finishTime=1452786538019, timeTaken=100, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=487063552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538001, Process_Duration=5, Process_End_Time=1452786538006, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786537935, firstTasksToStart=[ task_1452786537723_0001_1_00_000000 ], lastTaskFinishTime=1452786538016, lastTasksToFinish=[ task_1452786537723_0001_1_00_000000 ], minTaskDuration=81, maxTaskDuration=81, avgTaskDuration=81.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786537723_0001_1_00_000000 ], longestDurationTasks=[ task_1452786537723_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:58,021 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:58,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:58,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:58,024 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:58,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786537723_0001_00_000001, stoppedTime=1452786538024, exitStatus=0
2016-01-14 15:48:58,024 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:58,025 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B] attempt: attempt_1452786537723_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:58,025 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C58D06DE37714D9E9365AF34C305F85B
2016-01-14 15:48:58,025 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:58,025 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:58,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:58,026 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786537723_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:58,026 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:58,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:TASK_STARTED]: vertexName=C58D06DE37714D9E9365AF34C305F85B, taskId=task_1452786537723_0001_1_01_000000, scheduledTime=1452786538026, launchTime=1452786538026
2016-01-14 15:48:58,027 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537723_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:58,027 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:58,027 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:58,027 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE] comprehensivesort/(1/1) ...nsive/true_null_true_true, VertexName: C58D06DE37714D9E9365AF34C305F85B, VertexParallelism: 1, TaskAttemptID:attempt_1452786537723_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8AD4FA6AB68E47BA9B5CA68A0C3035FC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:58,027 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537723_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:58,027 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:58,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=487063552, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786538001, Process_Duration=5, Process_End_Time=1452786538006, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:58,028 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:58,028 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,028 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786537723_0001_00_000002
2016-01-14 15:48:58,029 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:58,029 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786537723_0001_00_000001
2016-01-14 15:48:58,029 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786537723_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:58,029 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786537723_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:58,029 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786537723_0001_00_000001, Exiting
2016-01-14 15:48:58,029 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786537723_0001_00_000001
2016-01-14 15:48:58,029 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786537723_0001_00_000001
2016-01-14 15:48:58,030 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786537723_0001_00_000002
2016-01-14 15:48:58,031 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:58,034 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786537723_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:58,034 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786537723_0001_00_000001 completed successfully
2016-01-14 15:48:58,034 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:58,036 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:58,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:58,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786537723_0001_00_000002, launchTime=1452786538036
2016-01-14 15:48:58,037 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786537723_0001_00_000002
2016-01-14 15:48:58,038 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786537723_0001_1_01_000000_0] to container: [container_1452786537723_0001_00_000002]
2016-01-14 15:48:58,038 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:58,039 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786537723_0001_1_01_000000_0] started. Is using containerId: [container_1452786537723_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:58,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C58D06DE37714D9E9365AF34C305F85B, taskAttemptId=attempt_1452786537723_0001_1_01_000000_0, startTime=1452786538038, containerId=container_1452786537723_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786537723_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:58,039 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537723_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:58,039 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537723_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:58,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:58,040 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786537723_0001_00_000002 given task: attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,040 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786537723_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58 Starting to run new task attempt: attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,040 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,040 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,041 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:58,041 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:58,041 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:58,041 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:58,041 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:58,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE] comprehensivesort/(1/1) ...nsive/true_null_true_true, VertexName: C58D06DE37714D9E9365AF34C305F85B, VertexParallelism: 1, TaskAttemptID:attempt_1452786537723_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8AD4FA6AB68E47BA9B5CA68A0C3035FC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:58,041 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:58,043 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:58,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,044 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786537723_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:58,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C43ABAFA369C41CCAA0DF9C8DF5E27C9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:58,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:58,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:58,048 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:58,048 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:58,048 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:58,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:58,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,052 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:58,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8AD4FA6AB68E47BA9B5CA68A0C3035FC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:58,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:58,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8AD4FA6AB68E47BA9B5CA68A0C3035FC
2016-01-14 15:48:58,054 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:58,055 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8AD4FA6AB68E47BA9B5CA68A0C3035FC
2016-01-14 15:48:58,063 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C58D06DE37714D9E9365AF34C305F85B, ordinal: 1
2016-01-14 15:48:58,063 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C58D06DE37714D9E9365AF34C305F85B, mem on start (mb), free: 213, total: 464, max: 672
2016-01-14 15:48:58,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:58,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:58,066 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:58,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8AD4FA6AB68E47BA9B5CA68A0C3035FC
2016-01-14 15:48:58,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:58,066 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:58,066 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:58,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:58,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:58,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:58,067 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:58,068 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C43ABAFA369C41CCAA0DF9C8DF5E27C9, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:58,068 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8AD4FA6AB68E47BA9B5CA68A0C3035FC, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:58,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:58,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:58,071 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:58,071 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:58,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C43ABAFA369C41CCAA0DF9C8DF5E27C9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:58,072 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C43ABAFA369C41CCAA0DF9C8DF5E27C9 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:58,072 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:58,072 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:58,072 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:58,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:58,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:58,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:58,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:58,075 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,078 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786537723_0001_1_00_000000_0_10113, runDuration: 59848, ]
2016-01-14 15:48:58,079 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:58,080 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: FF259DC655D646628D669E8206E61B56
2016-01-14 15:48:58,080 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true"], id: 8AD4FA6AB68E47BA9B5CA68A0C3035FC
2016-01-14 15:48:58,080 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true"]
2016-01-14 15:48:58,081 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] FF259DC655D646628D669E8206E61B56, for 1 inputs
2016-01-14 15:48:58,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786537723_0001_1_00_000000_0_10113] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786538083, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:58,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:58,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:58,083 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C43ABAFA369C41CCAA0DF9C8DF5E27C9] #20 in 1ms
2016-01-14 15:48:58,084 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C43ABAFA369C41CCAA0DF9C8DF5E27C9] #20, status:false, isInterrupted:false
2016-01-14 15:48:58,084 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:58,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:58,085 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:58,085 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:58,085 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:58,086 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:58,086 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:58,086 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C58D06DE37714D9E9365AF34C305F85B, all 1 inputs ready in: 00:00:00.005
2016-01-14 15:48:58,087 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@57118b17; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@40c2bfa9
2016-01-14 15:48:58,092 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:58,092 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:58,093 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,093 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786537723_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:58,094 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,094 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C58D06DE37714D9E9365AF34C305F85B, mem on close (mb), free: 210, total: 464, max: 672
2016-01-14 15:48:58,094 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C43ABAFA369C41CCAA0DF9C8DF5E27C9
2016-01-14 15:48:58,095 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:58,096 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:58,096 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:58,097 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:58,097 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:58 Completed running task attempt: attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,098 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786537723_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:58,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C58D06DE37714D9E9365AF34C305F85B, taskAttemptId=attempt_1452786537723_0001_1_01_000000_0, startTime=1452786538038, finishTime=1452786538099, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=487063552, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538086, Process_Duration=8, Process_End_Time=1452786538094, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:58,100 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786537723_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:58,100 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:58,100 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786537723_0001_1_01_000000_0
2016-01-14 15:48:58,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:TASK_FINISHED]: vertexName=C58D06DE37714D9E9365AF34C305F85B, taskId=task_1452786537723_0001_1_01_000000, startTime=1452786538038, finishTime=1452786538100, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452786537723_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=487063552, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538086, Process_Duration=8, Process_End_Time=1452786538094, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:58,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:58,101 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786537723_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:58,101 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B] : 1
2016-01-14 15:48:58,101 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:58,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:58,102 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:58,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:VERTEX_FINISHED]: vertexName=C58D06DE37714D9E9365AF34C305F85B, vertexId=vertex_1452786537723_0001_1_01, initRequestedTime=1452786537902, initedTime=1452786537914, startRequestedTime=1452786537920, startedTime=1452786537920, finishTime=1452786538101, timeTaken=181, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=487063552, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538086, Process_Duration=8, Process_End_Time=1452786538094, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786538038, firstTasksToStart=[ task_1452786537723_0001_1_01_000000 ], lastTaskFinishTime=1452786538099, lastTasksToFinish=[ task_1452786537723_0001_1_01_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786537723_0001_1_01_000000 ], longestDurationTasks=[ task_1452786537723_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:58,102 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:58,103 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:58,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786537723_0001_00_000002, stoppedTime=1452786538103, exitStatus=0
2016-01-14 15:48:58,103 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:58,103 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:58,104 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786537723_0001_1
2016-01-14 15:48:58,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:58,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786537723_0001_1
2016-01-14 15:48:58,105 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786537723_0001_1_00 [C43ABAFA369C41CCAA0DF9C8DF5E27C9]
2016-01-14 15:48:58,105 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8AD4FA6AB68E47BA9B5CA68A0C3035FC for vertex: vertex_1452786537723_0001_1_01 [C58D06DE37714D9E9365AF34C305F85B]
2016-01-14 15:48:58,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=487063552, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786538086, Process_Duration=8, Process_End_Time=1452786538094, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:58,110 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:58,110 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,111 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786537723_0001_00_000002
2016-01-14 15:48:58,111 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786537723_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:58,111 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786537723_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:58,111 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786537723_0001_00_000002, Exiting
2016-01-14 15:48:58,111 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786537723_0001_00_000002
2016-01-14 15:48:58,112 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786537723_0001_00_000002
2016-01-14 15:48:58,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786537723_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:58,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786537723_0001_00_000002 completed successfully
2016-01-14 15:48:58,120 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786537723_0001_1, queueSize=0
2016-01-14 15:48:58,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:58,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786537723_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786537723_0001_1, startTime=1452786537900, finishTime=1452786538118, timeTaken=218, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=859, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=974127104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573076087, Process_Duration=13, Process_End_Time=2905573076100, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:58,122 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786537723_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:58,122 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786537723_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:58,122 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786537723_0001_00_000002
2016-01-14 15:48:58,123 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786537723_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:58 Completed Dag: dag_1452786537723_0001_1
2016-01-14 15:48:58,124 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786537723_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:58,125 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786537723_0001_1_post
2016-01-14 15:48:58,125 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,125 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:58,128 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...nsive/true_null_true_true, applicationId=application_1452786537723_0001
2016-01-14 15:48:58,129 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:58,129 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:58,129 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:58,130 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:58,130 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:58,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:58,131 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:58,131 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:58,131 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:58,131 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:58,132 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:58,132 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:58,135 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:58,138 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BD9E3681C781470BAFC02538AAE57EEA/B2F2C6D9DDE34604B92A5625CE9861FE/.tez/application_1452786537723_0001
2016-01-14 15:48:58,139 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.431
2016-01-14 15:48:58,194 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:58,194 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:58,222 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 15:48:58,228 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 15:48:58,229 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:58,229 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:58,229 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:58,240 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:58,249 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true
2016-01-14 15:48:58,286 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:58,286 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:58,287 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true"]
2016-01-14 15:48:58,287 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:58,287 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:58,287 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:58,288 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ve/false_false_false_true
2016-01-14 15:48:58,288 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:58,297 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:58,302 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:58,303 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:58,303 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:58,311 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F/.tez/application_1452786538303_0001 doesn't exist and is created
2016-01-14 15:48:58,326 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:58,326 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:58,326 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F/.tez/application_1452786538303_0001
2016-01-14 15:48:58,329 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786538303_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:58,337 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:58,337 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:58,338 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:58,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:58,339 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F/.tez/application_1452786538303_0001/localmode-log-dir
2016-01-14 15:48:58,339 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F/.tez/application_1452786538303_0001/localmode-log-dir/history.txt.appattempt_1452786538303_0001_000000, maxErrors=10
2016-01-14 15:48:58,339 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:58,339 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786538303_0001_000000, appSubmitTime=1452786538329, launchTime=1452786538329
2016-01-14 15:48:58,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:58,345 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59897
2016-01-14 15:48:58,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:58,348 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:58,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:58,350 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:58,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786538303_0001_000000, startTime=1452786538352
2016-01-14 15:48:58,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:58,352 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:58,426 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:58,426 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:58,426 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_false_false_true, applicationId=application_1452786538303_0001, dagName=[06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F] comprehensivesort/(1/1) ...ve/false_false_false_true
2016-01-14 15:48:58,440 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F] comprehensivesort/(1/1) ...ve/false_false_false_true
2016-01-14 15:48:58,441 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786538303_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F/.tez/application_1452786538303_0001/localmode-log-dir/dag_1452786538303_0001_1.dot
2016-01-14 15:48:58,442 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F/.tez/application_1452786538303_0001/localmode-log-dir/dag_1452786538303_0001_1-tez-dag.pb.txt
2016-01-14 15:48:58,450 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786538303_0001_1
2016-01-14 15:48:58,450 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,450 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F] comprehensivesort/(1/1) ...ve/false_false_false_true
2016-01-14 15:48:58 Running Dag: dag_1452786538303_0001_1
2016-01-14 15:48:58,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786538303_0001_1, submitTime=1452786538440
2016-01-14 15:48:58,463 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:58,464 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AB35051DA6034BFB9F9DDD35D7FE782E
2016-01-14 15:48:58,464 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:58,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786538303_0001_1, initTime=1452786538463
2016-01-14 15:48:58,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:58,466 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786538303_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:58,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:DAG_STARTED]: dagID=dag_1452786538303_0001_1, startTime=1452786538466
2016-01-14 15:48:58,467 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:58,467 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786538303_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:58,467 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E24F9E22AFBC415E8506DB75D6199A6A : {34795A8FC76E474EB07E9667C5D60094={InputName=34795A8FC76E474EB07E9667C5D60094}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:58,467 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 34795A8FC76E474EB07E9667C5D60094, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:58,467 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,467 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,468 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,468 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]. Starting root input initializers: 1
2016-01-14 15:48:58,468 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_false_false_true, applicationId=application_1452786538303_0001, dagName=[06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F] comprehensivesort/(1/1) ...ve/false_false_false_true
2016-01-14 15:48:58,468 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:58,468 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 34795A8FC76E474EB07E9667C5D60094 on vertex vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,467 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:58,468 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,471 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:58,471 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,471 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,472 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,472 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6B84A534A8A44DBE8B2E63974120D0C9, vertexId=vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:58,472 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6B84A534A8A44DBE8B2E63974120D0C9, vertexId=vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,473 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:58,474 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 34795A8FC76E474EB07E9667C5D60094 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:58,474 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6B84A534A8A44DBE8B2E63974120D0C9, vertexName=AB35051DA6034BFB9F9DDD35D7FE782E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:58,474 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6B84A534A8A44DBE8B2E63974120D0C9, vertexId=vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,474 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:58,475 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:58,476 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:58,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AB35051DA6034BFB9F9DDD35D7FE782E, vertexId=vertex_1452786538303_0001_1_01, initRequestedTime=1452786538468, initedTime=1452786538475, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:58,476 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:58,477 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:58,477 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 34795A8FC76E474EB07E9667C5D60094 on vertex vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,477 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:58,478 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {34795A8FC76E474EB07E9667C5D60094=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:58,478 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] parallelism set to 1
2016-01-14 15:48:58,478 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:58,478 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786538303_0001_1_00, eventCount=1
2016-01-14 15:48:58,479 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E24F9E22AFBC415E8506DB75D6199A6A, vertexId=vertex_1452786538303_0001_1_00, initRequestedTime=1452786538467, initedTime=1452786538479, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:58,479 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] with distanceFromRoot: 0
2016-01-14 15:48:58,479 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:58,479 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:58,480 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,480 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,480 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786538303_0001_1_00, startRequestedTime=1452786538480, startedTime=1452786538480
2016-01-14 15:48:58,480 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:58,480 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:58,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:58,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:TASK_STARTED]: vertexName=E24F9E22AFBC415E8506DB75D6199A6A, taskId=task_1452786538303_0001_1_00_000000, scheduledTime=1452786538481, launchTime=1452786538481
2016-01-14 15:48:58,481 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538303_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:58,482 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786538303_0001_1_00 for vertex: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E] numStartedSources: 1 numSources: 1
2016-01-14 15:48:58,482 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E] with distanceFromRoot: 1
2016-01-14 15:48:58,482 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,483 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E24F9E22AFBC415E8506DB75D6199A6A in vertex: AB35051DA6034BFB9F9DDD35D7FE782E
2016-01-14 15:48:58,483 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AB35051DA6034BFB9F9DDD35D7FE782E with 1 source tasks and 1 pending tasks
2016-01-14 15:48:58,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786538303_0001_1_01, startRequestedTime=1452786538482, startedTime=1452786538482
2016-01-14 15:48:58,483 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786538303_0001, with dag id: dag_1452786538303_0001_1
2016-01-14 15:48:58,483 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:58,483 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:58,484 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F] comprehensivesort/(1/1) ...ve/false_false_false_true, VertexName: E24F9E22AFBC415E8506DB75D6199A6A, VertexParallelism: 1, TaskAttemptID:attempt_1452786538303_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=34795A8FC76E474EB07E9667C5D60094, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AB35051DA6034BFB9F9DDD35D7FE782E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:58,485 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538303_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:58,485 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:58,485 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:58,486 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:58,486 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:58,486 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786538303_0001_00_000001
2016-01-14 15:48:58,486 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:58,488 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786538303_0001_00_000001
2016-01-14 15:48:58,488 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:58,488 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:58,489 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:58,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786538303_0001_00_000001, launchTime=1452786538489
2016-01-14 15:48:58,490 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786538303_0001_00_000001
2016-01-14 15:48:58,490 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786538303_0001_1_00_000000_0] to container: [container_1452786538303_0001_00_000001]
2016-01-14 15:48:58,490 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:58,490 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786538303_0001_00_000001 given task: attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,490 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786538303_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,491 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786538303_0001_1_00_000000_0] started. Is using containerId: [container_1452786538303_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:58 Starting to run new task attempt: attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,491 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,491 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,491 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:58,491 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:58,491 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:58,491 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:58,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:58,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E24F9E22AFBC415E8506DB75D6199A6A, taskAttemptId=attempt_1452786538303_0001_1_00_000000_0, startTime=1452786538490, containerId=container_1452786538303_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786538303_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:58,492 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538303_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:58,492 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:58,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F] comprehensivesort/(1/1) ...ve/false_false_false_true, VertexName: E24F9E22AFBC415E8506DB75D6199A6A, VertexParallelism: 1, TaskAttemptID:attempt_1452786538303_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=34795A8FC76E474EB07E9667C5D60094, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AB35051DA6034BFB9F9DDD35D7FE782E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:58,492 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538303_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:58,492 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:58,493 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:58,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,494 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786538303_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:58,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AB35051DA6034BFB9F9DDD35D7FE782E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:58,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:58,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AB35051DA6034BFB9F9DDD35D7FE782E
2016-01-14 15:48:58,496 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:58,498 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:58,498 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AB35051DA6034BFB9F9DDD35D7FE782E
2016-01-14 15:48:58,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AB35051DA6034BFB9F9DDD35D7FE782E
2016-01-14 15:48:58,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=34795A8FC76E474EB07E9667C5D60094, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:58,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:58,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 34795A8FC76E474EB07E9667C5D60094
2016-01-14 15:48:58,500 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 34795A8FC76E474EB07E9667C5D60094
2016-01-14 15:48:58,502 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:58,502 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 34795A8FC76E474EB07E9667C5D60094
2016-01-14 15:48:58,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 34795A8FC76E474EB07E9667C5D60094
2016-01-14 15:48:58,503 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E24F9E22AFBC415E8506DB75D6199A6A, ordinal: 0
2016-01-14 15:48:58,503 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E24F9E22AFBC415E8506DB75D6199A6A, mem on start (mb), free: 127, total: 464, max: 672
2016-01-14 15:48:58,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:58,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:58,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:58,504 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:58,504 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:58,504 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:58,504 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:58,505 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:58,505 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:58,506 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AB35051DA6034BFB9F9DDD35D7FE782E, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:58,506 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 34795A8FC76E474EB07E9667C5D60094, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:58,506 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:58,506 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 34795A8FC76E474EB07E9667C5D60094 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:58,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:58,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:58,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 34795A8FC76E474EB07E9667C5D60094
2016-01-14 15:48:58,507 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:58,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 34795A8FC76E474EB07E9667C5D60094
2016-01-14 15:48:58,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:58,508 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,508 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:58,508 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@36dfe178
2016-01-14 15:48:58,509 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@36dfe178
2016-01-14 15:48:58,509 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:58,509 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:58,511 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:58,513 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 34795A8FC76E474EB07E9667C5D60094
2016-01-14 15:48:58,513 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: C08ADBEC03B445FDA8FBE49C666C845F
2016-01-14 15:48:58,513 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] C08ADBEC03B445FDA8FBE49C666C845F
2016-01-14 15:48:58,514 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@73ceafd9; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@67b0af5; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@780c5e25; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:58,514 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:58,515 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:58,515 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:58,535 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:58,535 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:58,536 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:58,536 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:58,536 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:58,538 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:58,538 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:58,538 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E24F9E22AFBC415E8506DB75D6199A6A, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:58,543 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,543 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E24F9E22AFBC415E8506DB75D6199A6A, mem on close (mb), free: 226, total: 474, max: 672
2016-01-14 15:48:58,543 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:58,544 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:58,544 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:58,544 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:58,547 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:58 Completed running task attempt: attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,550 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786538303_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:58,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786538303_0001_1_00, eventCount=1
2016-01-14 15:48:58,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786538538, Process_Duration=5, Process_End_Time=1452786538543, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:58,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:58,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E24F9E22AFBC415E8506DB75D6199A6A, taskAttemptId=attempt_1452786538303_0001_1_00_000000_0, startTime=1452786538490, finishTime=1452786538551, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538538, Process_Duration=5, Process_End_Time=1452786538543, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:58,551 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538303_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:58,551 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:58,551 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:58,552 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,551 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786538303_0001_1_00_000000_0
2016-01-14 15:48:58,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:58,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:TASK_FINISHED]: vertexName=E24F9E22AFBC415E8506DB75D6199A6A, taskId=task_1452786538303_0001_1_00_000000, startTime=1452786538490, finishTime=1452786538552, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452786538303_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538538, Process_Duration=5, Process_End_Time=1452786538543, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:58,552 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538303_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:58,553 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] : 1
2016-01-14 15:48:58,553 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,553 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786538303_0001_00_000001
2016-01-14 15:48:58,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:VERTEX_FINISHED]: vertexName=E24F9E22AFBC415E8506DB75D6199A6A, vertexId=vertex_1452786538303_0001_1_00, initRequestedTime=1452786538467, initedTime=1452786538479, startRequestedTime=1452786538480, startedTime=1452786538480, finishTime=1452786538553, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538538, Process_Duration=5, Process_End_Time=1452786538543, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786538490, firstTasksToStart=[ task_1452786538303_0001_1_00_000000 ], lastTaskFinishTime=1452786538551, lastTasksToFinish=[ task_1452786538303_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786538303_0001_1_00_000000 ], longestDurationTasks=[ task_1452786538303_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:58,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:58,554 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:58,554 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:58,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786538303_0001_00_000001, stoppedTime=1452786538554, exitStatus=0
2016-01-14 15:48:58,554 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:58,554 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E] attempt: attempt_1452786538303_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:58,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:58,554 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AB35051DA6034BFB9F9DDD35D7FE782E
2016-01-14 15:48:58,555 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,555 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:58,555 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786538303_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:58,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:TASK_STARTED]: vertexName=AB35051DA6034BFB9F9DDD35D7FE782E, taskId=task_1452786538303_0001_1_01_000000, scheduledTime=1452786538555, launchTime=1452786538555
2016-01-14 15:48:58,555 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:58,555 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538303_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:58,555 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,555 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:58,555 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F] comprehensivesort/(1/1) ...ve/false_false_false_true, VertexName: AB35051DA6034BFB9F9DDD35D7FE782E, VertexParallelism: 1, TaskAttemptID:attempt_1452786538303_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E24F9E22AFBC415E8506DB75D6199A6A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6B84A534A8A44DBE8B2E63974120D0C9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:58,556 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538303_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:58,556 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:58,556 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786538303_0001_00_000002
2016-01-14 15:48:58,556 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:58,558 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786538303_0001_00_000002
2016-01-14 15:48:58,558 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:58,559 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:58,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:58,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786538303_0001_00_000002, launchTime=1452786538559
2016-01-14 15:48:58,560 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786538303_0001_00_000002
2016-01-14 15:48:58,560 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786538303_0001_1_01_000000_0] to container: [container_1452786538303_0001_00_000002]
2016-01-14 15:48:58,561 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:58,561 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786538303_0001_1_01_000000_0] started. Is using containerId: [container_1452786538303_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:58,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AB35051DA6034BFB9F9DDD35D7FE782E, taskAttemptId=attempt_1452786538303_0001_1_01_000000_0, startTime=1452786538561, containerId=container_1452786538303_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786538303_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:58,561 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:58,561 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786538303_0001_00_000002 given task: attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,561 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538303_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:58,562 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538303_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:58,562 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786538303_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58 Starting to run new task attempt: attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,563 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,563 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,563 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:58,563 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:58,563 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:58,564 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:58,564 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:58,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F] comprehensivesort/(1/1) ...ve/false_false_false_true, VertexName: AB35051DA6034BFB9F9DDD35D7FE782E, VertexParallelism: 1, TaskAttemptID:attempt_1452786538303_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E24F9E22AFBC415E8506DB75D6199A6A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6B84A534A8A44DBE8B2E63974120D0C9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:58,564 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:58,565 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,565 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786538303_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:58,565 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786538303_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:58,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:58,565 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786538303_0001_00_000001, Exiting
2016-01-14 15:48:58,565 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786538303_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:58,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,565 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786538303_0001_00_000001
2016-01-14 15:48:58,566 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786538303_0001_00_000001
2016-01-14 15:48:58,567 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786538303_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:58,567 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786538303_0001_00_000001 completed successfully
2016-01-14 15:48:58,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,567 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:58,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E24F9E22AFBC415E8506DB75D6199A6A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:58,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:58,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,569 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:58,569 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:58,570 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,570 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:58,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6B84A534A8A44DBE8B2E63974120D0C9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:58,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:58,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6B84A534A8A44DBE8B2E63974120D0C9
2016-01-14 15:48:58,574 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:58,574 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6B84A534A8A44DBE8B2E63974120D0C9
2016-01-14 15:48:58,577 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AB35051DA6034BFB9F9DDD35D7FE782E, ordinal: 1
2016-01-14 15:48:58,578 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AB35051DA6034BFB9F9DDD35D7FE782E, mem on start (mb), free: 221, total: 474, max: 672
2016-01-14 15:48:58,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:58,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:58,583 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:58,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6B84A534A8A44DBE8B2E63974120D0C9
2016-01-14 15:48:58,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:58,584 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:58,584 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:58,584 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:58,584 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:58,584 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:58,585 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:58,585 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E24F9E22AFBC415E8506DB75D6199A6A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:58,585 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6B84A534A8A44DBE8B2E63974120D0C9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:58,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:58,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E24F9E22AFBC415E8506DB75D6199A6A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:58,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:58,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:58,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,586 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:58,586 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:58,586 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E24F9E22AFBC415E8506DB75D6199A6A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:58,587 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:58,587 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:58,587 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:58,588 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,588 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:58,588 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,589 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786538303_0001_1_00_000000_0_10118, runDuration: 53053, ]
2016-01-14 15:48:58,590 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786538303_0001_1_00_000000_0_10118] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786538590, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:58,590 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,591 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:58,591 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:58,591 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E24F9E22AFBC415E8506DB75D6199A6A] #21 in 2ms
2016-01-14 15:48:58,591 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: C08ADBEC03B445FDA8FBE49C666C845F
2016-01-14 15:48:58,591 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E24F9E22AFBC415E8506DB75D6199A6A] #21, status:false, isInterrupted:false
2016-01-14 15:48:58,592 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true"], id: 6B84A534A8A44DBE8B2E63974120D0C9
2016-01-14 15:48:58,592 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true"]
2016-01-14 15:48:58,592 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] C08ADBEC03B445FDA8FBE49C666C845F, for 1 inputs
2016-01-14 15:48:58,593 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:58,593 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:58,593 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:58,593 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:58,593 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:58,593 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,593 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AB35051DA6034BFB9F9DDD35D7FE782E, all 1 inputs ready in: 00:00:00.001
2016-01-14 15:48:58,593 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:58,595 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@4e445c52; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6a0933a2
2016-01-14 15:48:58,600 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:58,600 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:58,601 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,601 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786538303_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:58,601 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,602 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AB35051DA6034BFB9F9DDD35D7FE782E, mem on close (mb), free: 218, total: 474, max: 672
2016-01-14 15:48:58,602 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E24F9E22AFBC415E8506DB75D6199A6A
2016-01-14 15:48:58,602 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:58,602 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:58,602 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:58,603 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:58,603 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:58 Completed running task attempt: attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,604 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786538303_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:58,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:58,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AB35051DA6034BFB9F9DDD35D7FE782E, taskAttemptId=attempt_1452786538303_0001_1_01_000000_0, startTime=1452786538561, finishTime=1452786538605, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538594, Process_Duration=7, Process_End_Time=1452786538601, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:58,605 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538303_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:58,605 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786538303_0001_1_01_000000_0
2016-01-14 15:48:58,606 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:58,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:58,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:TASK_FINISHED]: vertexName=AB35051DA6034BFB9F9DDD35D7FE782E, taskId=task_1452786538303_0001_1_01_000000, startTime=1452786538561, finishTime=1452786538606, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452786538303_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538594, Process_Duration=7, Process_End_Time=1452786538601, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:58,606 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538303_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:58,606 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E] : 1
2016-01-14 15:48:58,606 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:58,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:VERTEX_FINISHED]: vertexName=AB35051DA6034BFB9F9DDD35D7FE782E, vertexId=vertex_1452786538303_0001_1_01, initRequestedTime=1452786538468, initedTime=1452786538475, startRequestedTime=1452786538482, startedTime=1452786538482, finishTime=1452786538607, timeTaken=125, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786538594, Process_Duration=7, Process_End_Time=1452786538601, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786538561, firstTasksToStart=[ task_1452786538303_0001_1_01_000000 ], lastTaskFinishTime=1452786538605, lastTasksToFinish=[ task_1452786538303_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786538303_0001_1_01_000000 ], longestDurationTasks=[ task_1452786538303_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:58,608 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:58,608 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:58,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786538303_0001_00_000002, stoppedTime=1452786538608, exitStatus=0
2016-01-14 15:48:58,609 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:58,609 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:58,609 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786538303_0001_1
2016-01-14 15:48:58,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786538303_0001_1
2016-01-14 15:48:58,609 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6B84A534A8A44DBE8B2E63974120D0C9 for vertex: vertex_1452786538303_0001_1_01 [AB35051DA6034BFB9F9DDD35D7FE782E]
2016-01-14 15:48:58,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:58,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786538594, Process_Duration=7, Process_End_Time=1452786538601, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:58,612 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:58,612 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,612 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786538303_0001_00_000002
2016-01-14 15:48:58,612 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786538303_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:58,612 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786538303_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:58,612 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786538303_0001_00_000002, Exiting
2016-01-14 15:48:58,612 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786538303_0001_00_000002
2016-01-14 15:48:58,612 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786538303_0001_00_000002
2016-01-14 15:48:58,613 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786538303_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:58,613 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786538303_0001_00_000002 completed successfully
2016-01-14 15:48:58,616 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786538303_0001_1_00 [E24F9E22AFBC415E8506DB75D6199A6A]
2016-01-14 15:48:58,618 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786538303_0001_1, queueSize=0
2016-01-14 15:48:58,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538303_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786538303_0001_1, startTime=1452786538466, finishTime=1452786538616, timeTaken=150, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=1518, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=994050048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573077132, Process_Duration=12, Process_End_Time=2905573077144, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:58,619 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786538303_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:58,619 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786538303_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:58,619 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538303_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:58,619 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786538303_0001_00_000002
2016-01-14 15:48:58,620 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:58 Completed Dag: dag_1452786538303_0001_1
2016-01-14 15:48:58,621 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786538303_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:58,621 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786538303_0001_1_post
2016-01-14 15:48:58,622 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,622 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:58,626 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_false_false_true, applicationId=application_1452786538303_0001
2016-01-14 15:48:58,627 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:58,627 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:58,627 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:58,628 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:58,628 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:58,628 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:58,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:58,628 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:58,628 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:58,629 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:58,629 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:58,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:58,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:58,634 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/06EF6BAB0B8E40219D1442D9A23089B9/D9FC2E8FEA244E8299D08E98F8E2735F/.tez/application_1452786538303_0001
2016-01-14 15:48:58,634 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.348
2016-01-14 15:48:58,688 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:58,688 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:58,718 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 15:48:58,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 15:48:58,723 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:58,723 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:58,723 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:58,734 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:58,740 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true
2016-01-14 15:48:58,774 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:58,774 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:58,774 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true"]
2016-01-14 15:48:58,774 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:58,775 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:58,775 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:58,776 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/true_false_false_true
2016-01-14 15:48:58,776 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:58,785 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:58,789 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:58,789 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:58,789 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:58,798 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7/.tez/application_1452786538789_0001 doesn't exist and is created
2016-01-14 15:48:58,813 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:58,813 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:58,814 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7/.tez/application_1452786538789_0001
2016-01-14 15:48:58,817 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786538789_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:58,825 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:58,825 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:58,827 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:58,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:58,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7/.tez/application_1452786538789_0001/localmode-log-dir
2016-01-14 15:48:58,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7/.tez/application_1452786538789_0001/localmode-log-dir/history.txt.appattempt_1452786538789_0001_000000, maxErrors=10
2016-01-14 15:48:58,828 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:58,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786538789_0001_000000, appSubmitTime=1452786538816, launchTime=1452786538816
2016-01-14 15:48:58,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:58,832 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:58,832 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:58,832 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:58,835 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59373
2016-01-14 15:48:58,836 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:58,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786538789_0001_000000, startTime=1452786538838
2016-01-14 15:48:58,838 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:58,838 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:58,913 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:58,914 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:58,914 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_false_false_true, applicationId=application_1452786538789_0001, dagName=[3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7] comprehensivesort/(1/1) ...ive/true_false_false_true
2016-01-14 15:48:58,925 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7] comprehensivesort/(1/1) ...ive/true_false_false_true
2016-01-14 15:48:58,926 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786538789_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7/.tez/application_1452786538789_0001/localmode-log-dir/dag_1452786538789_0001_1.dot
2016-01-14 15:48:58,927 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7/.tez/application_1452786538789_0001/localmode-log-dir/dag_1452786538789_0001_1-tez-dag.pb.txt
2016-01-14 15:48:58,933 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786538789_0001_1
2016-01-14 15:48:58,933 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,933 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7] comprehensivesort/(1/1) ...ive/true_false_false_true
2016-01-14 15:48:58 Running Dag: dag_1452786538789_0001_1
2016-01-14 15:48:58,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786538789_0001_1, submitTime=1452786538925
2016-01-14 15:48:58,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:58,945 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D2372DE59EBD4BD2BF897840FC58753F
2016-01-14 15:48:58,945 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:58,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786538789_0001_1, initTime=1452786538944
2016-01-14 15:48:58,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:58,946 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786538789_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:58,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:DAG_STARTED]: dagID=dag_1452786538789_0001_1, startTime=1452786538946
2016-01-14 15:48:58,946 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:58,946 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786538789_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:58,947 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FE2CB27A531449008525424611755D2F : {7565A6D19D9D4EB996752F8EB2EC63E0={InputName=7565A6D19D9D4EB996752F8EB2EC63E0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:58,947 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7565A6D19D9D4EB996752F8EB2EC63E0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:58,947 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:58,947 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:58,947 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:58,947 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]. Starting root input initializers: 1
2016-01-14 15:48:58,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:58,948 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:58,948 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7565A6D19D9D4EB996752F8EB2EC63E0 on vertex vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:58,948 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_false_false_true, applicationId=application_1452786538789_0001, dagName=[3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7] comprehensivesort/(1/1) ...ive/true_false_false_true
2016-01-14 15:48:58,950 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:58,950 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7565A6D19D9D4EB996752F8EB2EC63E0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:58,950 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:58,951 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:58,952 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:58,952 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7565A6D19D9D4EB996752F8EB2EC63E0 on vertex vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:58,948 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:58,955 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:58,955 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:58,956 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:58,956 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:58,956 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F6EA29B6EAE34FA3ACE2B45C9E75BA00, vertexId=vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:58,956 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F6EA29B6EAE34FA3ACE2B45C9E75BA00, vertexId=vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:58,958 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F6EA29B6EAE34FA3ACE2B45C9E75BA00, vertexName=D2372DE59EBD4BD2BF897840FC58753F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:58,958 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F6EA29B6EAE34FA3ACE2B45C9E75BA00, vertexId=vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:58,959 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:58,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D2372DE59EBD4BD2BF897840FC58753F, vertexId=vertex_1452786538789_0001_1_01, initRequestedTime=1452786538948, initedTime=1452786538959, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:58,959 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:58,960 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:58,960 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7565A6D19D9D4EB996752F8EB2EC63E0=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:58,960 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] parallelism set to 1
2016-01-14 15:48:58,960 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:58,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786538789_0001_1_00, eventCount=1
2016-01-14 15:48:58,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE2CB27A531449008525424611755D2F, vertexId=vertex_1452786538789_0001_1_00, initRequestedTime=1452786538947, initedTime=1452786538961, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:58,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:58,961 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] with distanceFromRoot: 0
2016-01-14 15:48:58,961 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:58,961 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FE2CB27A531449008525424611755D2F
2016-01-14 15:48:58,962 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:58,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786538789_0001_1_00, startRequestedTime=1452786538961, startedTime=1452786538961
2016-01-14 15:48:58,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:58,962 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:58,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:58,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:TASK_STARTED]: vertexName=FE2CB27A531449008525424611755D2F, taskId=task_1452786538789_0001_1_00_000000, scheduledTime=1452786538962, launchTime=1452786538962
2016-01-14 15:48:58,963 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786538789_0001, with dag id: dag_1452786538789_0001_1
2016-01-14 15:48:58,963 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538789_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:58,963 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786538789_0001_1_00 for vertex: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F] numStartedSources: 1 numSources: 1
2016-01-14 15:48:58,964 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F] with distanceFromRoot: 1
2016-01-14 15:48:58,964 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:58,966 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FE2CB27A531449008525424611755D2F in vertex: D2372DE59EBD4BD2BF897840FC58753F
2016-01-14 15:48:58,967 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D2372DE59EBD4BD2BF897840FC58753F with 1 source tasks and 1 pending tasks
2016-01-14 15:48:58,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786538789_0001_1_01, startRequestedTime=1452786538964, startedTime=1452786538964
2016-01-14 15:48:58,967 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:58,967 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7] comprehensivesort/(1/1) ...ive/true_false_false_true, VertexName: FE2CB27A531449008525424611755D2F, VertexParallelism: 1, TaskAttemptID:attempt_1452786538789_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7565A6D19D9D4EB996752F8EB2EC63E0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D2372DE59EBD4BD2BF897840FC58753F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:58,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:58,968 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538789_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:58,968 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:58,969 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:58,970 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:58,970 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:58,970 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786538789_0001_00_000001
2016-01-14 15:48:58,970 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:58,971 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786538789_0001_00_000001
2016-01-14 15:48:58,971 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:58,972 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:58,972 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:58,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786538789_0001_00_000001, launchTime=1452786538972
2016-01-14 15:48:58,973 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786538789_0001_00_000001
2016-01-14 15:48:58,973 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786538789_0001_1_00_000000_0] to container: [container_1452786538789_0001_00_000001]
2016-01-14 15:48:58,973 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:58,973 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786538789_0001_1_00_000000_0] started. Is using containerId: [container_1452786538789_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:58,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE2CB27A531449008525424611755D2F, taskAttemptId=attempt_1452786538789_0001_1_00_000000_0, startTime=1452786538973, containerId=container_1452786538789_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786538789_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:58,974 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538789_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:58,974 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538789_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:58,974 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:58,974 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786538789_0001_00_000001 given task: attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:58,974 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786538789_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:58 Starting to run new task attempt: attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:58,975 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:58,975 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:58,975 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:58,975 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:58,975 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:58,975 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:58,975 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:58,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7] comprehensivesort/(1/1) ...ive/true_false_false_true, VertexName: FE2CB27A531449008525424611755D2F, VertexParallelism: 1, TaskAttemptID:attempt_1452786538789_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7565A6D19D9D4EB996752F8EB2EC63E0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D2372DE59EBD4BD2BF897840FC58753F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:58,976 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:58,976 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:58,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:58,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,976 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786538789_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:58,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7565A6D19D9D4EB996752F8EB2EC63E0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:58,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:58,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7565A6D19D9D4EB996752F8EB2EC63E0
2016-01-14 15:48:58,977 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7565A6D19D9D4EB996752F8EB2EC63E0
2016-01-14 15:48:58,979 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:58,979 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7565A6D19D9D4EB996752F8EB2EC63E0
2016-01-14 15:48:58,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7565A6D19D9D4EB996752F8EB2EC63E0
2016-01-14 15:48:58,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D2372DE59EBD4BD2BF897840FC58753F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:58,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:58,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D2372DE59EBD4BD2BF897840FC58753F
2016-01-14 15:48:58,981 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:58,982 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D2372DE59EBD4BD2BF897840FC58753F
2016-01-14 15:48:58,982 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:58,983 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:58,983 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:58,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D2372DE59EBD4BD2BF897840FC58753F
2016-01-14 15:48:58,987 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE2CB27A531449008525424611755D2F, ordinal: 0
2016-01-14 15:48:58,987 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE2CB27A531449008525424611755D2F, mem on start (mb), free: 136, total: 474, max: 672
2016-01-14 15:48:58,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:58,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:58,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:58,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:58,988 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:58,988 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:58,988 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:58,988 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:58,988 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:58,988 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:58,989 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7565A6D19D9D4EB996752F8EB2EC63E0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:58,989 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D2372DE59EBD4BD2BF897840FC58753F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:58,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:58,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7565A6D19D9D4EB996752F8EB2EC63E0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:58,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:58,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:58,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7565A6D19D9D4EB996752F8EB2EC63E0
2016-01-14 15:48:58,989 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:58,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7565A6D19D9D4EB996752F8EB2EC63E0
2016-01-14 15:48:58,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:58,990 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:58,990 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:58,990 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3f8d5031
2016-01-14 15:48:58,991 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3f8d5031
2016-01-14 15:48:58,992 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:58,992 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:58,993 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:58,993 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 7565A6D19D9D4EB996752F8EB2EC63E0
2016-01-14 15:48:58,993 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: E4C2CA8D9D4248C5AE2550EFF510C900
2016-01-14 15:48:58,993 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] E4C2CA8D9D4248C5AE2550EFF510C900
2016-01-14 15:48:58,995 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@11a05d8; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5375a878; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@19392d2b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:58,995 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:58,995 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:58,995 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:59,015 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:59,015 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:59,015 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:59,015 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:59,015 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:59,016 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:59,016 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:59,016 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE2CB27A531449008525424611755D2F, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:59,020 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:59,020 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE2CB27A531449008525424611755D2F, mem on close (mb), free: 224, total: 474, max: 672
2016-01-14 15:48:59,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:59,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:59,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:59,021 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:59,023 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:59 Completed running task attempt: attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:59,026 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786538789_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:59,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786538789_0001_1_00, eventCount=1
2016-01-14 15:48:59,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE2CB27A531449008525424611755D2F, taskAttemptId=attempt_1452786538789_0001_1_00_000000_0, startTime=1452786538973, finishTime=1452786539026, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539016, Process_Duration=4, Process_End_Time=1452786539020, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:59,027 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538789_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:59,027 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:59,027 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786538789_0001_1_00_000000_0
2016-01-14 15:48:59,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:TASK_FINISHED]: vertexName=FE2CB27A531449008525424611755D2F, taskId=task_1452786538789_0001_1_00_000000, startTime=1452786538973, finishTime=1452786539027, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452786538789_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539016, Process_Duration=4, Process_End_Time=1452786539020, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:59,028 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538789_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:59,028 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] : 1
2016-01-14 15:48:59,028 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:59,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786539016, Process_Duration=4, Process_End_Time=1452786539020, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:59,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:VERTEX_FINISHED]: vertexName=FE2CB27A531449008525424611755D2F, vertexId=vertex_1452786538789_0001_1_00, initRequestedTime=1452786538947, initedTime=1452786538961, startRequestedTime=1452786538961, startedTime=1452786538961, finishTime=1452786539028, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539016, Process_Duration=4, Process_End_Time=1452786539020, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786538973, firstTasksToStart=[ task_1452786538789_0001_1_00_000000 ], lastTaskFinishTime=1452786539026, lastTasksToFinish=[ task_1452786538789_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786538789_0001_1_00_000000 ], longestDurationTasks=[ task_1452786538789_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:59,029 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:59,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:59,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:59,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:59,029 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:59,030 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:59,029 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786538789_0001_00_000001, stoppedTime=1452786539030, exitStatus=0
2016-01-14 15:48:59,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:59,030 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786538789_0001_00_000001
2016-01-14 15:48:59,030 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:59,031 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F] attempt: attempt_1452786538789_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:59,031 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786538789_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:59,032 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D2372DE59EBD4BD2BF897840FC58753F
2016-01-14 15:48:59,032 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:59,032 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786538789_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:59,032 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:59,032 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786538789_0001_00_000001, Exiting
2016-01-14 15:48:59,032 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786538789_0001_00_000001
2016-01-14 15:48:59,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:TASK_STARTED]: vertexName=D2372DE59EBD4BD2BF897840FC58753F, taskId=task_1452786538789_0001_1_01_000000, scheduledTime=1452786539032, launchTime=1452786539032
2016-01-14 15:48:59,033 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538789_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:59,032 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786538789_0001_00_000001
2016-01-14 15:48:59,033 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:59,033 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7] comprehensivesort/(1/1) ...ive/true_false_false_true, VertexName: D2372DE59EBD4BD2BF897840FC58753F, VertexParallelism: 1, TaskAttemptID:attempt_1452786538789_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE2CB27A531449008525424611755D2F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F6EA29B6EAE34FA3ACE2B45C9E75BA00, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:59,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:59,034 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538789_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:59,034 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:59,035 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786538789_0001_00_000002
2016-01-14 15:48:59,035 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:59,035 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786538789_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:59,035 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786538789_0001_00_000001 completed successfully
2016-01-14 15:48:59,036 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786538789_0001_00_000001
2016-01-14 15:48:59,036 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:59,037 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786538789_0001_00_000002
2016-01-14 15:48:59,038 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:59,040 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:59,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:59,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786538789_0001_00_000002, launchTime=1452786539040
2016-01-14 15:48:59,041 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786538789_0001_00_000002
2016-01-14 15:48:59,042 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786538789_0001_1_01_000000_0] to container: [container_1452786538789_0001_00_000002]
2016-01-14 15:48:59,043 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:59,043 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786538789_0001_00_000002 given task: attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,043 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786538789_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,043 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786538789_0001_1_01_000000_0] started. Is using containerId: [container_1452786538789_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:59,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D2372DE59EBD4BD2BF897840FC58753F, taskAttemptId=attempt_1452786538789_0001_1_01_000000_0, startTime=1452786539043, containerId=container_1452786538789_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786538789_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:59 Starting to run new task attempt: attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,043 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:59,043 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538789_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:59,044 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538789_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:59,043 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,044 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,044 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:59,044 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:59,045 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:59,045 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:59,045 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:59,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7] comprehensivesort/(1/1) ...ive/true_false_false_true, VertexName: D2372DE59EBD4BD2BF897840FC58753F, VertexParallelism: 1, TaskAttemptID:attempt_1452786538789_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE2CB27A531449008525424611755D2F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F6EA29B6EAE34FA3ACE2B45C9E75BA00, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:59,045 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:59,045 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:59,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FE2CB27A531449008525424611755D2F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:59,046 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786538789_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:59,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F6EA29B6EAE34FA3ACE2B45C9E75BA00, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:59,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:59,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FE2CB27A531449008525424611755D2F
2016-01-14 15:48:59,049 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:59,049 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:59,049 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FE2CB27A531449008525424611755D2F
2016-01-14 15:48:59,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FE2CB27A531449008525424611755D2F
2016-01-14 15:48:59,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:59,050 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:59,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F6EA29B6EAE34FA3ACE2B45C9E75BA00
2016-01-14 15:48:59,050 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:59,050 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F6EA29B6EAE34FA3ACE2B45C9E75BA00
2016-01-14 15:48:59,057 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D2372DE59EBD4BD2BF897840FC58753F, ordinal: 1
2016-01-14 15:48:59,057 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D2372DE59EBD4BD2BF897840FC58753F, mem on start (mb), free: 218, total: 474, max: 672
2016-01-14 15:48:59,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:59,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:59,060 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:59,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F6EA29B6EAE34FA3ACE2B45C9E75BA00
2016-01-14 15:48:59,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:59,060 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:59,061 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:59,061 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:59,061 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:59,061 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:59,061 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:59,062 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FE2CB27A531449008525424611755D2F, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:59,062 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F6EA29B6EAE34FA3ACE2B45C9E75BA00, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:59,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:59,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FE2CB27A531449008525424611755D2F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:59,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FE2CB27A531449008525424611755D2F
2016-01-14 15:48:59,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:59,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:59,063 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:59,063 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:59,063 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FE2CB27A531449008525424611755D2F with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:59,064 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:59,064 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:59,064 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:59,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FE2CB27A531449008525424611755D2F
2016-01-14 15:48:59,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:59,065 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,066 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786538789_0001_1_00_000000_0_10125, runDuration: 44637, ]
2016-01-14 15:48:59,068 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:59,068 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: E4C2CA8D9D4248C5AE2550EFF510C900
2016-01-14 15:48:59,068 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true"], id: F6EA29B6EAE34FA3ACE2B45C9E75BA00
2016-01-14 15:48:59,069 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true"]
2016-01-14 15:48:59,069 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] E4C2CA8D9D4248C5AE2550EFF510C900, for 1 inputs
2016-01-14 15:48:59,069 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786538789_0001_1_00_000000_0_10125] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786539069, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:59,070 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FE2CB27A531449008525424611755D2F
2016-01-14 15:48:59,070 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:59,070 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FE2CB27A531449008525424611755D2F] #22 in 2ms
2016-01-14 15:48:59,070 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FE2CB27A531449008525424611755D2F] #22, status:false, isInterrupted:false
2016-01-14 15:48:59,071 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:59,072 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:59,072 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:59,072 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:59,073 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:59,073 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FE2CB27A531449008525424611755D2F
2016-01-14 15:48:59,073 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:59,073 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D2372DE59EBD4BD2BF897840FC58753F, all 1 inputs ready in: 00:00:00.004
2016-01-14 15:48:59,074 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@30e31e7d; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@ee6b99e
2016-01-14 15:48:59,078 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:59,078 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:59,078 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,079 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786538789_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:59,079 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,079 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D2372DE59EBD4BD2BF897840FC58753F, mem on close (mb), free: 215, total: 474, max: 672
2016-01-14 15:48:59,080 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FE2CB27A531449008525424611755D2F
2016-01-14 15:48:59,080 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:59,080 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:59,081 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:59,081 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:59,081 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:59 Completed running task attempt: attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,082 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786538789_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:59,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786539073, Process_Duration=6, Process_End_Time=1452786539079, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:59,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D2372DE59EBD4BD2BF897840FC58753F, taskAttemptId=attempt_1452786538789_0001_1_01_000000_0, startTime=1452786539043, finishTime=1452786539082, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539073, Process_Duration=6, Process_End_Time=1452786539079, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:59,083 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:59,083 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:59,083 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786538789_0001_00_000002
2016-01-14 15:48:59,083 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786538789_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:59,084 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786538789_0001_1_01_000000_0
2016-01-14 15:48:59,084 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:59,084 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:59,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:TASK_FINISHED]: vertexName=D2372DE59EBD4BD2BF897840FC58753F, taskId=task_1452786538789_0001_1_01_000000, startTime=1452786539043, finishTime=1452786539084, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1452786538789_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539073, Process_Duration=6, Process_End_Time=1452786539079, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:59,084 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786538789_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:59,084 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F] : 1
2016-01-14 15:48:59,085 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:59,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:VERTEX_FINISHED]: vertexName=D2372DE59EBD4BD2BF897840FC58753F, vertexId=vertex_1452786538789_0001_1_01, initRequestedTime=1452786538948, initedTime=1452786538959, startRequestedTime=1452786538964, startedTime=1452786538964, finishTime=1452786539085, timeTaken=121, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539073, Process_Duration=6, Process_End_Time=1452786539079, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786539043, firstTasksToStart=[ task_1452786538789_0001_1_01_000000 ], lastTaskFinishTime=1452786539082, lastTasksToFinish=[ task_1452786538789_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786538789_0001_1_01_000000 ], longestDurationTasks=[ task_1452786538789_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:59,085 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:59,086 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:59,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786538789_0001_00_000002, stoppedTime=1452786539086, exitStatus=0
2016-01-14 15:48:59,086 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:59,086 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:59,086 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786538789_0001_1
2016-01-14 15:48:59,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786538789_0001_1
2016-01-14 15:48:59,086 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F6EA29B6EAE34FA3ACE2B45C9E75BA00 for vertex: vertex_1452786538789_0001_1_01 [D2372DE59EBD4BD2BF897840FC58753F]
2016-01-14 15:48:59,087 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:59,088 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:59,092 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786538789_0001_1_00 [FE2CB27A531449008525424611755D2F]
2016-01-14 15:48:59,094 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786538789_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:59,094 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786538789_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:59,094 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786538789_0001_00_000002, Exiting
2016-01-14 15:48:59,094 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786538789_0001_00_000002
2016-01-14 15:48:59,094 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786538789_0001_00_000002
2016-01-14 15:48:59,094 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786538789_0001_1, queueSize=0
2016-01-14 15:48:59,095 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786538789_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:59,095 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:59,095 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786538789_0001_00_000002 completed successfully
2016-01-14 15:48:59,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786538789_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786538789_0001_1, startTime=1452786538946, finishTime=1452786539092, timeTaken=146, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=994050048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573078089, Process_Duration=10, Process_End_Time=2905573078099, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:59,096 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786538789_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:59,096 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786538789_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:59,096 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786538789_0001_00_000002
2016-01-14 15:48:59,097 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786538789_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:59 Completed Dag: dag_1452786538789_0001_1
2016-01-14 15:48:59,098 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786538789_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:59,098 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786538789_0001_1_post
2016-01-14 15:48:59,098 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,098 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:59,101 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_false_false_true, applicationId=application_1452786538789_0001
2016-01-14 15:48:59,102 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:59,103 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:59,103 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:59,103 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:59,103 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:59,103 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:59,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:59,103 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:59,104 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:59,104 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:59,104 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:59,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:59,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:59,108 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CCDC22352A3461AB512839771E1359F/D30E3554AAF04BEC8E1E4DB6118D74B7/.tez/application_1452786538789_0001
2016-01-14 15:48:59,109 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.335
2016-01-14 15:48:59,160 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:59,160 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:59,193 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 15:48:59,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:48:59,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:59,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:59,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:59,212 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:59,218 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true
2016-01-14 15:48:59,253 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:59,254 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:59,254 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true"]
2016-01-14 15:48:59,254 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:59,254 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:59,254 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:59,255 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...sive/true_true_false_true
2016-01-14 15:48:59,255 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:59,263 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:59,267 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:59,268 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:59,268 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:59,276 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B/.tez/application_1452786539268_0001 doesn't exist and is created
2016-01-14 15:48:59,290 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:59,290 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:59,290 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B/.tez/application_1452786539268_0001
2016-01-14 15:48:59,293 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786539268_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:59,300 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:59,301 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:59,302 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:59,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:59,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B/.tez/application_1452786539268_0001/localmode-log-dir
2016-01-14 15:48:59,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B/.tez/application_1452786539268_0001/localmode-log-dir/history.txt.appattempt_1452786539268_0001_000000, maxErrors=10
2016-01-14 15:48:59,303 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:59,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786539268_0001_000000, appSubmitTime=1452786539292, launchTime=1452786539292
2016-01-14 15:48:59,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:59,310 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40265
2016-01-14 15:48:59,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:59,313 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:59,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:59,313 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:59,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786539268_0001_000000, startTime=1452786539315
2016-01-14 15:48:59,315 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:59,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:59,390 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:59,390 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:59,390 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_true_false_true, applicationId=application_1452786539268_0001, dagName=[3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B] comprehensivesort/(1/1) ...sive/true_true_false_true
2016-01-14 15:48:59,402 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B] comprehensivesort/(1/1) ...sive/true_true_false_true
2016-01-14 15:48:59,404 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786539268_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B/.tez/application_1452786539268_0001/localmode-log-dir/dag_1452786539268_0001_1.dot
2016-01-14 15:48:59,404 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B/.tez/application_1452786539268_0001/localmode-log-dir/dag_1452786539268_0001_1-tez-dag.pb.txt
2016-01-14 15:48:59,411 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786539268_0001_1
2016-01-14 15:48:59,411 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,411 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B] comprehensivesort/(1/1) ...sive/true_true_false_true
2016-01-14 15:48:59 Running Dag: dag_1452786539268_0001_1
2016-01-14 15:48:59,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786539268_0001_1, submitTime=1452786539403
2016-01-14 15:48:59,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:59,421 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C10B193395D546B0A232B8461945E699
2016-01-14 15:48:59,422 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:59,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786539268_0001_1, initTime=1452786539421
2016-01-14 15:48:59,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:59,422 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786539268_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:59,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:DAG_STARTED]: dagID=dag_1452786539268_0001_1, startTime=1452786539423
2016-01-14 15:48:59,423 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:59,423 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:59,423 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786539268_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:59,424 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FBB75DF9096343B18021DB74CB20CA2A : {C1EC515F08C94B6BA0EF6957DF3FD669={InputName=C1EC515F08C94B6BA0EF6957DF3FD669}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:59,424 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C1EC515F08C94B6BA0EF6957DF3FD669, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:59,424 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,424 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,424 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,425 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]. Starting root input initializers: 1
2016-01-14 15:48:59,424 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_true_false_true, applicationId=application_1452786539268_0001, dagName=[3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B] comprehensivesort/(1/1) ...sive/true_true_false_true
2016-01-14 15:48:59,425 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C1EC515F08C94B6BA0EF6957DF3FD669 on vertex vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,425 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:59,426 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,427 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:59,428 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C1EC515F08C94B6BA0EF6957DF3FD669 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:59,428 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:59,428 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:59,429 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:59,429 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,429 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,429 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,429 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=DFF0AC8066C34152AC2793D2F108F9B1, vertexId=vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:59,429 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=DFF0AC8066C34152AC2793D2F108F9B1, vertexId=vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,431 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:59,431 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C1EC515F08C94B6BA0EF6957DF3FD669 on vertex vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,431 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=DFF0AC8066C34152AC2793D2F108F9B1, vertexName=C10B193395D546B0A232B8461945E699, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:59,431 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=DFF0AC8066C34152AC2793D2F108F9B1, vertexId=vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:59,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C10B193395D546B0A232B8461945E699, vertexId=vertex_1452786539268_0001_1_01, initRequestedTime=1452786539426, initedTime=1452786539432, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:59,433 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:59,433 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:59,433 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C1EC515F08C94B6BA0EF6957DF3FD669=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:59,434 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] parallelism set to 1
2016-01-14 15:48:59,434 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:59,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786539268_0001_1_00, eventCount=1
2016-01-14 15:48:59,434 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:59,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FBB75DF9096343B18021DB74CB20CA2A, vertexId=vertex_1452786539268_0001_1_00, initRequestedTime=1452786539424, initedTime=1452786539434, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:59,435 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] with distanceFromRoot: 0
2016-01-14 15:48:59,435 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:59,435 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,436 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:59,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786539268_0001_1_00, startRequestedTime=1452786539435, startedTime=1452786539435
2016-01-14 15:48:59,436 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:59,437 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:59,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:TASK_STARTED]: vertexName=FBB75DF9096343B18021DB74CB20CA2A, taskId=task_1452786539268_0001_1_00_000000, scheduledTime=1452786539437, launchTime=1452786539437
2016-01-14 15:48:59,437 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539268_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:59,438 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786539268_0001_1_00 for vertex: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699] numStartedSources: 1 numSources: 1
2016-01-14 15:48:59,438 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699] with distanceFromRoot: 1
2016-01-14 15:48:59,438 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,438 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FBB75DF9096343B18021DB74CB20CA2A in vertex: C10B193395D546B0A232B8461945E699
2016-01-14 15:48:59,439 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C10B193395D546B0A232B8461945E699 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:59,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:59,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786539268_0001_1_01, startRequestedTime=1452786539438, startedTime=1452786539438
2016-01-14 15:48:59,439 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:59,440 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B] comprehensivesort/(1/1) ...sive/true_true_false_true, VertexName: FBB75DF9096343B18021DB74CB20CA2A, VertexParallelism: 1, TaskAttemptID:attempt_1452786539268_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1EC515F08C94B6BA0EF6957DF3FD669, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C10B193395D546B0A232B8461945E699, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:59,440 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539268_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:59,440 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:59,441 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:59,441 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:59,441 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:59,442 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786539268_0001_00_000001
2016-01-14 15:48:59,442 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:59,444 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786539268_0001_00_000001
2016-01-14 15:48:59,444 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:59,444 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:59,444 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:59,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786539268_0001_00_000001, launchTime=1452786539444
2016-01-14 15:48:59,445 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786539268_0001_00_000001
2016-01-14 15:48:59,446 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786539268_0001_1_00_000000_0] to container: [container_1452786539268_0001_00_000001]
2016-01-14 15:48:59,446 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:59,447 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786539268_0001_1_00_000000_0] started. Is using containerId: [container_1452786539268_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:59,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FBB75DF9096343B18021DB74CB20CA2A, taskAttemptId=attempt_1452786539268_0001_1_00_000000_0, startTime=1452786539447, containerId=container_1452786539268_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786539268_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:59,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:59,447 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786539268_0001_00_000001 given task: attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59,447 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539268_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:59,448 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539268_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:59,448 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786539268_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59 Starting to run new task attempt: attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59,448 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59,449 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,449 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:59,449 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:59,449 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:59,449 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:59,449 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786539268_0001, with dag id: dag_1452786539268_0001_1
2016-01-14 15:48:59,449 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:59,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B] comprehensivesort/(1/1) ...sive/true_true_false_true, VertexName: FBB75DF9096343B18021DB74CB20CA2A, VertexParallelism: 1, TaskAttemptID:attempt_1452786539268_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1EC515F08C94B6BA0EF6957DF3FD669, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C10B193395D546B0A232B8461945E699, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:59,453 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:59,454 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:59,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,454 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786539268_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:59,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C1EC515F08C94B6BA0EF6957DF3FD669, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:59,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C10B193395D546B0A232B8461945E699, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:59,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:59,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C1EC515F08C94B6BA0EF6957DF3FD669
2016-01-14 15:48:59,456 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C1EC515F08C94B6BA0EF6957DF3FD669
2016-01-14 15:48:59,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:59,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C10B193395D546B0A232B8461945E699
2016-01-14 15:48:59,458 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:59,458 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:59,458 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C1EC515F08C94B6BA0EF6957DF3FD669
2016-01-14 15:48:59,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C1EC515F08C94B6BA0EF6957DF3FD669
2016-01-14 15:48:59,459 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:59,459 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C10B193395D546B0A232B8461945E699
2016-01-14 15:48:59,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C10B193395D546B0A232B8461945E699
2016-01-14 15:48:59,465 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FBB75DF9096343B18021DB74CB20CA2A, ordinal: 0
2016-01-14 15:48:59,465 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FBB75DF9096343B18021DB74CB20CA2A, mem on start (mb), free: 133, total: 474, max: 672
2016-01-14 15:48:59,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:59,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:59,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:59,465 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:59,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:59,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:59,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:59,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:59,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:59,466 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C1EC515F08C94B6BA0EF6957DF3FD669, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:59,466 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C10B193395D546B0A232B8461945E699, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:59,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:59,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C1EC515F08C94B6BA0EF6957DF3FD669
2016-01-14 15:48:59,467 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:59,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C1EC515F08C94B6BA0EF6957DF3FD669
2016-01-14 15:48:59,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C1EC515F08C94B6BA0EF6957DF3FD669 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:59,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:59,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:59,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:59,468 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59,469 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:59,469 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7b08f3f4
2016-01-14 15:48:59,471 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7b08f3f4
2016-01-14 15:48:59,471 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:59,472 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:59,472 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:59,473 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: C1EC515F08C94B6BA0EF6957DF3FD669
2016-01-14 15:48:59,473 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: E2BB246BB7E243A79DEB4E95028D11DC
2016-01-14 15:48:59,473 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] E2BB246BB7E243A79DEB4E95028D11DC
2016-01-14 15:48:59,476 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@34516931; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4cd12acc; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@3850de7a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:59,476 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:59,476 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:59,477 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:59,499 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:59,499 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:59,499 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:59,499 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:59,500 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:59,500 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:59,501 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:59,502 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FBB75DF9096343B18021DB74CB20CA2A, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:59,506 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59,507 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FBB75DF9096343B18021DB74CB20CA2A, mem on close (mb), free: 220, total: 474, max: 672
2016-01-14 15:48:59,507 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:59,507 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:59,507 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:59,507 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:59,511 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:59 Completed running task attempt: attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59,514 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786539268_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:59,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786539268_0001_1_00, eventCount=1
2016-01-14 15:48:59,515 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:59,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FBB75DF9096343B18021DB74CB20CA2A, taskAttemptId=attempt_1452786539268_0001_1_00_000000_0, startTime=1452786539447, finishTime=1452786539514, timeTaken=67, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539502, Process_Duration=4, Process_End_Time=1452786539506, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:59,515 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539268_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:59,515 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:59,515 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786539268_0001_1_00_000000_0
2016-01-14 15:48:59,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:TASK_FINISHED]: vertexName=FBB75DF9096343B18021DB74CB20CA2A, taskId=task_1452786539268_0001_1_00_000000, startTime=1452786539447, finishTime=1452786539517, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1452786539268_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539502, Process_Duration=4, Process_End_Time=1452786539506, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:59,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:59,517 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539268_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:59,517 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] : 1
2016-01-14 15:48:59,517 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786539502, Process_Duration=4, Process_End_Time=1452786539506, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:59,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:VERTEX_FINISHED]: vertexName=FBB75DF9096343B18021DB74CB20CA2A, vertexId=vertex_1452786539268_0001_1_00, initRequestedTime=1452786539424, initedTime=1452786539434, startRequestedTime=1452786539435, startedTime=1452786539435, finishTime=1452786539517, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539502, Process_Duration=4, Process_End_Time=1452786539506, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786539447, firstTasksToStart=[ task_1452786539268_0001_1_00_000000 ], lastTaskFinishTime=1452786539514, lastTasksToFinish=[ task_1452786539268_0001_1_00_000000 ], minTaskDuration=67, maxTaskDuration=67, avgTaskDuration=67.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786539268_0001_1_00_000000 ], longestDurationTasks=[ task_1452786539268_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:59,518 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:59,518 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,519 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786539268_0001_00_000001
2016-01-14 15:48:59,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:59,518 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:59,519 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:59,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786539268_0001_00_000001, stoppedTime=1452786539519, exitStatus=0
2016-01-14 15:48:59,519 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:59,520 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699] attempt: attempt_1452786539268_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:59,520 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C10B193395D546B0A232B8461945E699
2016-01-14 15:48:59,520 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,520 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:59,520 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786539268_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:59,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:TASK_STARTED]: vertexName=C10B193395D546B0A232B8461945E699, taskId=task_1452786539268_0001_1_01_000000, scheduledTime=1452786539520, launchTime=1452786539520
2016-01-14 15:48:59,521 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539268_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:59,521 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,521 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:59,520 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:59,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:59,521 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B] comprehensivesort/(1/1) ...sive/true_true_false_true, VertexName: C10B193395D546B0A232B8461945E699, VertexParallelism: 1, TaskAttemptID:attempt_1452786539268_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FBB75DF9096343B18021DB74CB20CA2A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DFF0AC8066C34152AC2793D2F108F9B1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:59,521 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539268_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:59,522 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:59,522 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786539268_0001_00_000002
2016-01-14 15:48:59,523 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:59,524 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786539268_0001_00_000002
2016-01-14 15:48:59,524 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:59,525 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:59,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786539268_0001_00_000002, launchTime=1452786539524
2016-01-14 15:48:59,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:59,525 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786539268_0001_00_000002
2016-01-14 15:48:59,526 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786539268_0001_1_01_000000_0] to container: [container_1452786539268_0001_00_000002]
2016-01-14 15:48:59,526 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:59,526 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786539268_0001_00_000002 given task: attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,526 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786539268_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,526 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786539268_0001_1_01_000000_0] started. Is using containerId: [container_1452786539268_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:59 Starting to run new task attempt: attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,527 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:59,527 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,527 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:59,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C10B193395D546B0A232B8461945E699, taskAttemptId=attempt_1452786539268_0001_1_01_000000_0, startTime=1452786539526, containerId=container_1452786539268_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786539268_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:59,527 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:59,527 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539268_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:59,527 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:59,528 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:59,528 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539268_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:59,528 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:59,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B] comprehensivesort/(1/1) ...sive/true_true_false_true, VertexName: C10B193395D546B0A232B8461945E699, VertexParallelism: 1, TaskAttemptID:attempt_1452786539268_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FBB75DF9096343B18021DB74CB20CA2A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DFF0AC8066C34152AC2793D2F108F9B1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:59,528 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:59,529 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:59,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,529 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786539268_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:59,529 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786539268_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:59,529 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786539268_0001_00_000001, Exiting
2016-01-14 15:48:59,529 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786539268_0001_00_000001
2016-01-14 15:48:59,529 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786539268_0001_00_000001
2016-01-14 15:48:59,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FBB75DF9096343B18021DB74CB20CA2A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:59,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:59,531 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DFF0AC8066C34152AC2793D2F108F9B1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:59,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,530 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786539268_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:59,532 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:59,532 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:59,533 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,533 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:59,534 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786539268_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:59,534 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786539268_0001_00_000001 completed successfully
2016-01-14 15:48:59,534 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:59,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:59,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DFF0AC8066C34152AC2793D2F108F9B1
2016-01-14 15:48:59,536 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:59,536 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: DFF0AC8066C34152AC2793D2F108F9B1
2016-01-14 15:48:59,538 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C10B193395D546B0A232B8461945E699, ordinal: 1
2016-01-14 15:48:59,538 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C10B193395D546B0A232B8461945E699, mem on start (mb), free: 215, total: 474, max: 672
2016-01-14 15:48:59,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:59,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:59,546 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:59,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DFF0AC8066C34152AC2793D2F108F9B1
2016-01-14 15:48:59,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:59,546 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:59,546 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:59,546 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:59,546 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:59,547 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:59,547 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:59,547 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FBB75DF9096343B18021DB74CB20CA2A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:59,547 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DFF0AC8066C34152AC2793D2F108F9B1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:59,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:59,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FBB75DF9096343B18021DB74CB20CA2A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:59,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:59,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:59,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,548 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:59,548 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:59,549 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FBB75DF9096343B18021DB74CB20CA2A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:59,549 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:59,549 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:59,549 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:59,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:59,552 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,553 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786539268_0001_1_00_000000_0_10131, runDuration: 54522, ]
2016-01-14 15:48:59,554 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:59,555 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: E2BB246BB7E243A79DEB4E95028D11DC
2016-01-14 15:48:59,555 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true"], id: DFF0AC8066C34152AC2793D2F108F9B1
2016-01-14 15:48:59,555 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786539268_0001_1_00_000000_0_10131] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786539555, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:59,555 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true"]
2016-01-14 15:48:59,555 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] E2BB246BB7E243A79DEB4E95028D11DC, for 1 inputs
2016-01-14 15:48:59,555 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,556 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:59,557 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FBB75DF9096343B18021DB74CB20CA2A] #23 in 3ms
2016-01-14 15:48:59,557 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FBB75DF9096343B18021DB74CB20CA2A] #23, status:false, isInterrupted:false
2016-01-14 15:48:59,558 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:59,559 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:59,559 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:59,559 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:59,560 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:59,560 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,560 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C10B193395D546B0A232B8461945E699, all 1 inputs ready in: 00:00:00.005
2016-01-14 15:48:59,560 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:59,561 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@3d65dec9; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6b057d9c
2016-01-14 15:48:59,565 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:59,565 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:59,565 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,566 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786539268_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:59,566 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,566 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C10B193395D546B0A232B8461945E699, mem on close (mb), free: 212, total: 474, max: 672
2016-01-14 15:48:59,566 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FBB75DF9096343B18021DB74CB20CA2A
2016-01-14 15:48:59,567 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:59,567 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:59,567 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:59,568 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:59,568 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:59 Completed running task attempt: attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,569 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786539268_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:59,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786539560, Process_Duration=6, Process_End_Time=1452786539566, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:59,570 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:48:59,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C10B193395D546B0A232B8461945E699, taskAttemptId=attempt_1452786539268_0001_1_01_000000_0, startTime=1452786539526, finishTime=1452786539570, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539560, Process_Duration=6, Process_End_Time=1452786539566, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:59,570 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:59,570 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539268_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:59,570 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786539268_0001_00_000002
2016-01-14 15:48:59,571 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786539268_0001_1_01_000000_0
2016-01-14 15:48:59,571 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:59,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:59,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:TASK_FINISHED]: vertexName=C10B193395D546B0A232B8461945E699, taskId=task_1452786539268_0001_1_01_000000, startTime=1452786539526, finishTime=1452786539572, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452786539268_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539560, Process_Duration=6, Process_End_Time=1452786539566, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:59,572 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539268_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:59,572 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699] : 1
2016-01-14 15:48:59,572 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,573 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:VERTEX_FINISHED]: vertexName=C10B193395D546B0A232B8461945E699, vertexId=vertex_1452786539268_0001_1_01, initRequestedTime=1452786539426, initedTime=1452786539432, startRequestedTime=1452786539438, startedTime=1452786539438, finishTime=1452786539572, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539560, Process_Duration=6, Process_End_Time=1452786539566, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786539526, firstTasksToStart=[ task_1452786539268_0001_1_01_000000 ], lastTaskFinishTime=1452786539570, lastTasksToFinish=[ task_1452786539268_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786539268_0001_1_01_000000 ], longestDurationTasks=[ task_1452786539268_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:59,573 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:59,573 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:59,574 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:59,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786539268_0001_00_000002, stoppedTime=1452786539574, exitStatus=0
2016-01-14 15:48:59,574 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:59,574 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:59,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:59,574 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786539268_0001_1
2016-01-14 15:48:59,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786539268_0001_1
2016-01-14 15:48:59,574 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786539268_0001_1_00 [FBB75DF9096343B18021DB74CB20CA2A]
2016-01-14 15:48:59,574 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: DFF0AC8066C34152AC2793D2F108F9B1 for vertex: vertex_1452786539268_0001_1_01 [C10B193395D546B0A232B8461945E699]
2016-01-14 15:48:59,582 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786539268_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:59,582 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786539268_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:59,582 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786539268_0001_00_000002, Exiting
2016-01-14 15:48:59,582 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786539268_0001_00_000002
2016-01-14 15:48:59,582 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786539268_0001_00_000002
2016-01-14 15:48:59,582 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786539268_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:59,582 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786539268_0001_00_000002 completed successfully
2016-01-14 15:48:59,583 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786539268_0001_1, queueSize=0
2016-01-14 15:48:59,583 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:59,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539268_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786539268_0001_1, startTime=1452786539423, finishTime=1452786539581, timeTaken=158, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=994050048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573079062, Process_Duration=10, Process_End_Time=2905573079072, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:59,584 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786539268_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:59,584 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786539268_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:59,584 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786539268_0001_00_000002
2016-01-14 15:48:59,584 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539268_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:59 Completed Dag: dag_1452786539268_0001_1
2016-01-14 15:48:59,585 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786539268_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:59,585 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786539268_0001_1_post
2016-01-14 15:48:59,585 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,586 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:59,590 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_true_false_true, applicationId=application_1452786539268_0001
2016-01-14 15:48:59,592 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:59,592 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:59,592 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:59,592 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:59,592 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:59,592 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:59,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:59,593 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:59,593 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:59,593 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:59,593 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:59,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:59,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:59,599 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3CE1D1B7E81A43199A833BBD828F4309/D9D2927D3B1D434CA856833AF506170B/.tez/application_1452786539268_0001
2016-01-14 15:48:59,600 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.346
2016-01-14 15:48:59,650 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:59,650 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:59,678 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 15:48:59,683 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 15:48:59,683 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:59,683 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:59,684 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:59,694 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:59,700 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true
2016-01-14 15:48:59,734 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:59,734 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:59,734 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true"]
2016-01-14 15:48:59,734 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:48:59,735 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:48:59,735 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:48:59,735 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/false_true_false_true
2016-01-14 15:48:59,736 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:59,750 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:59,755 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:59,755 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:59,755 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:59,764 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23/.tez/application_1452786539755_0001 doesn't exist and is created
2016-01-14 15:48:59,778 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:59,778 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:59,779 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23/.tez/application_1452786539755_0001
2016-01-14 15:48:59,783 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786539755_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:59,790 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:59,791 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:59,792 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:59,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:59,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23/.tez/application_1452786539755_0001/localmode-log-dir
2016-01-14 15:48:59,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23/.tez/application_1452786539755_0001/localmode-log-dir/history.txt.appattempt_1452786539755_0001_000000, maxErrors=10
2016-01-14 15:48:59,792 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:59,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786539755_0001_000000, appSubmitTime=1452786539782, launchTime=1452786539782
2016-01-14 15:48:59,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:59,795 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:59,798 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:59,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:59,801 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53470
2016-01-14 15:48:59,803 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:59,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786539755_0001_000000, startTime=1452786539804
2016-01-14 15:48:59,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:59,804 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:59,880 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:59,880 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:59,880 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_true_false_true, applicationId=application_1452786539755_0001, dagName=[0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23] comprehensivesort/(1/1) ...ive/false_true_false_true
2016-01-14 15:48:59,892 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23] comprehensivesort/(1/1) ...ive/false_true_false_true
2016-01-14 15:48:59,893 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786539755_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23/.tez/application_1452786539755_0001/localmode-log-dir/dag_1452786539755_0001_1.dot
2016-01-14 15:48:59,894 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23/.tez/application_1452786539755_0001/localmode-log-dir/dag_1452786539755_0001_1-tez-dag.pb.txt
2016-01-14 15:48:59,902 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786539755_0001_1
2016-01-14 15:48:59,902 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,902 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23] comprehensivesort/(1/1) ...ive/false_true_false_true
2016-01-14 15:48:59 Running Dag: dag_1452786539755_0001_1
2016-01-14 15:48:59,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786539755_0001_1, submitTime=1452786539892
2016-01-14 15:48:59,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:59,915 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BFADFC004048489C9A7038D694BD14D8
2016-01-14 15:48:59,916 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:59,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786539755_0001_1, initTime=1452786539915
2016-01-14 15:48:59,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:59,916 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786539755_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:59,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:DAG_STARTED]: dagID=dag_1452786539755_0001_1, startTime=1452786539916
2016-01-14 15:48:59,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:59,917 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:59,917 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786539755_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:59,917 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ABBD1A0EAF9E467C859C8AE9EAB31EF2 : {DB8E9842810649D4B1CB62D03992718B={InputName=DB8E9842810649D4B1CB62D03992718B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:59,917 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: DB8E9842810649D4B1CB62D03992718B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:59,917 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:48:59,917 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_true_false_true, applicationId=application_1452786539755_0001, dagName=[0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23] comprehensivesort/(1/1) ...ive/false_true_false_true
2016-01-14 15:48:59,918 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:48:59,918 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:48:59,918 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]. Starting root input initializers: 1
2016-01-14 15:48:59,919 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:59,919 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:48:59,921 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: DB8E9842810649D4B1CB62D03992718B on vertex vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:48:59,922 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:59,922 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:48:59,922 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:48:59,923 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:48:59,923 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=20BA619F8EA943AC803713A6347D4878, vertexId=vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:59,923 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=20BA619F8EA943AC803713A6347D4878, vertexId=vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:48:59,924 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:59,924 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input DB8E9842810649D4B1CB62D03992718B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:59,925 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:59,925 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:59,925 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=20BA619F8EA943AC803713A6347D4878, vertexName=BFADFC004048489C9A7038D694BD14D8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:59,925 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=20BA619F8EA943AC803713A6347D4878, vertexId=vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:48:59,927 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:59,927 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: DB8E9842810649D4B1CB62D03992718B on vertex vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:48:59,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BFADFC004048489C9A7038D694BD14D8, vertexId=vertex_1452786539755_0001_1_01, initRequestedTime=1452786539919, initedTime=1452786539929, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:59,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:59,929 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:59,930 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:59,930 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {DB8E9842810649D4B1CB62D03992718B=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:59,931 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] parallelism set to 1
2016-01-14 15:48:59,931 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:59,931 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786539755_0001, with dag id: dag_1452786539755_0001_1
2016-01-14 15:48:59,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786539755_0001_1_00, eventCount=1
2016-01-14 15:48:59,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, vertexId=vertex_1452786539755_0001_1_00, initRequestedTime=1452786539917, initedTime=1452786539932, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:59,932 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] with distanceFromRoot: 0
2016-01-14 15:48:59,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:59,932 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:59,932 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:48:59,933 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:48:59,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786539755_0001_1_00, startRequestedTime=1452786539932, startedTime=1452786539932
2016-01-14 15:48:59,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:59,933 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:59,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:59,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:TASK_STARTED]: vertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, taskId=task_1452786539755_0001_1_00_000000, scheduledTime=1452786539933, launchTime=1452786539933
2016-01-14 15:48:59,934 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539755_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:59,934 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786539755_0001_1_00 for vertex: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8] numStartedSources: 1 numSources: 1
2016-01-14 15:48:59,934 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8] with distanceFromRoot: 1
2016-01-14 15:48:59,935 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:48:59,935 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ABBD1A0EAF9E467C859C8AE9EAB31EF2 in vertex: BFADFC004048489C9A7038D694BD14D8
2016-01-14 15:48:59,936 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BFADFC004048489C9A7038D694BD14D8 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:59,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786539755_0001_1_01, startRequestedTime=1452786539935, startedTime=1452786539935
2016-01-14 15:48:59,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:59,936 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:59,936 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23] comprehensivesort/(1/1) ...ive/false_true_false_true, VertexName: ABBD1A0EAF9E467C859C8AE9EAB31EF2, VertexParallelism: 1, TaskAttemptID:attempt_1452786539755_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DB8E9842810649D4B1CB62D03992718B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BFADFC004048489C9A7038D694BD14D8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:59,937 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539755_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:59,937 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:59,938 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:59,938 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:59,938 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:59,939 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786539755_0001_00_000001
2016-01-14 15:48:59,939 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:59,940 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786539755_0001_00_000001
2016-01-14 15:48:59,940 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:59,941 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:59,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786539755_0001_00_000001, launchTime=1452786539941
2016-01-14 15:48:59,941 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:59,942 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786539755_0001_00_000001
2016-01-14 15:48:59,942 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786539755_0001_1_00_000000_0] to container: [container_1452786539755_0001_00_000001]
2016-01-14 15:48:59,942 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:59,942 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786539755_0001_00_000001 given task: attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:48:59,942 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786539755_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:48:59,943 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786539755_0001_1_00_000000_0] started. Is using containerId: [container_1452786539755_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:59,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:59,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, taskAttemptId=attempt_1452786539755_0001_1_00_000000_0, startTime=1452786539942, containerId=container_1452786539755_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786539755_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:59 Starting to run new task attempt: attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:48:59,943 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539755_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:59,943 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:48:59,943 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539755_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:59,944 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:59,944 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:59,944 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:59,944 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:59,944 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:59,945 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:59,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23] comprehensivesort/(1/1) ...ive/false_true_false_true, VertexName: ABBD1A0EAF9E467C859C8AE9EAB31EF2, VertexParallelism: 1, TaskAttemptID:attempt_1452786539755_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DB8E9842810649D4B1CB62D03992718B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BFADFC004048489C9A7038D694BD14D8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:59,945 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:59,946 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:48:59,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:59,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,946 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786539755_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:59,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DB8E9842810649D4B1CB62D03992718B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:59,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:59,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DB8E9842810649D4B1CB62D03992718B
2016-01-14 15:48:59,947 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: DB8E9842810649D4B1CB62D03992718B
2016-01-14 15:48:59,948 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:59,949 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:59,949 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: DB8E9842810649D4B1CB62D03992718B
2016-01-14 15:48:59,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DB8E9842810649D4B1CB62D03992718B
2016-01-14 15:48:59,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BFADFC004048489C9A7038D694BD14D8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:59,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:59,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BFADFC004048489C9A7038D694BD14D8
2016-01-14 15:48:59,955 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ABBD1A0EAF9E467C859C8AE9EAB31EF2, ordinal: 0
2016-01-14 15:48:59,955 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ABBD1A0EAF9E467C859C8AE9EAB31EF2, mem on start (mb), free: 130, total: 474, max: 672
2016-01-14 15:48:59,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:59,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:59,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:59,955 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:59,956 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BFADFC004048489C9A7038D694BD14D8
2016-01-14 15:48:59,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BFADFC004048489C9A7038D694BD14D8
2016-01-14 15:48:59,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:59,956 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:59,957 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:59,957 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:59,957 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:59,957 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:59,957 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:59,957 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DB8E9842810649D4B1CB62D03992718B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:59,957 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BFADFC004048489C9A7038D694BD14D8, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:59,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:59,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DB8E9842810649D4B1CB62D03992718B
2016-01-14 15:48:59,958 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:59,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DB8E9842810649D4B1CB62D03992718B
2016-01-14 15:48:59,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DB8E9842810649D4B1CB62D03992718B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:59,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:59,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:59,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:59,958 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:48:59,960 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:59,960 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@716b0b73
2016-01-14 15:48:59,965 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:59,965 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: DB8E9842810649D4B1CB62D03992718B
2016-01-14 15:48:59,966 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 751FDB18FB6347718CAB35423E3E9404
2016-01-14 15:48:59,966 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 751FDB18FB6347718CAB35423E3E9404
2016-01-14 15:48:59,966 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@716b0b73
2016-01-14 15:48:59,966 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:59,966 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:59,969 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@7e0f1732; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@10bc1446; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@2bd7083f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:59,970 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:59,970 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:59,970 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:59,991 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:59,991 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:59,992 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:59,992 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:59,992 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:59,993 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:59,993 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:59,993 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ABBD1A0EAF9E467C859C8AE9EAB31EF2, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:59,998 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:48:59,998 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ABBD1A0EAF9E467C859C8AE9EAB31EF2, mem on close (mb), free: 207, total: 463, max: 672
2016-01-14 15:48:59,998 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:59,998 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:59,998 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:59,999 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:49:00,001 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:49:00 Completed running task attempt: attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:49:00,004 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786539755_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:00,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786539755_0001_1_00, eventCount=1
2016-01-14 15:49:00,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=486014976, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786539993, Process_Duration=5, Process_End_Time=1452786539998, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:49:00,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:00,006 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:00,006 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,006 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786539755_0001_00_000001
2016-01-14 15:49:00,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, taskAttemptId=attempt_1452786539755_0001_1_00_000000_0, startTime=1452786539942, finishTime=1452786540005, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=486014976, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539993, Process_Duration=5, Process_End_Time=1452786539998, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:00,006 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539755_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:00,006 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786539755_0001_1_00_000000_0
2016-01-14 15:49:00,007 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:00,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:TASK_FINISHED]: vertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, taskId=task_1452786539755_0001_1_00_000000, startTime=1452786539942, finishTime=1452786540007, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1452786539755_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=486014976, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539993, Process_Duration=5, Process_End_Time=1452786539998, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:00,007 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539755_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:00,008 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] : 1
2016-01-14 15:49:00,008 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:49:00,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:00,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:VERTEX_FINISHED]: vertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, vertexId=vertex_1452786539755_0001_1_00, initRequestedTime=1452786539917, initedTime=1452786539932, startRequestedTime=1452786539932, startedTime=1452786539932, finishTime=1452786540008, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=486014976, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786539993, Process_Duration=5, Process_End_Time=1452786539998, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786539942, firstTasksToStart=[ task_1452786539755_0001_1_00_000000 ], lastTaskFinishTime=1452786540005, lastTasksToFinish=[ task_1452786539755_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786539755_0001_1_00_000000 ], longestDurationTasks=[ task_1452786539755_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:00,009 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:00,009 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:00,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786539755_0001_00_000001, stoppedTime=1452786540009, exitStatus=0
2016-01-14 15:49:00,009 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:00,009 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8] attempt: attempt_1452786539755_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:49:00,009 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BFADFC004048489C9A7038D694BD14D8
2016-01-14 15:49:00,010 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:49:00,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:00,010 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:00,010 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786539755_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:49:00,010 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:00,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:TASK_STARTED]: vertexName=BFADFC004048489C9A7038D694BD14D8, taskId=task_1452786539755_0001_1_01_000000, scheduledTime=1452786540010, launchTime=1452786540010
2016-01-14 15:49:00,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:00,011 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539755_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:00,011 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:49:00,011 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:49:00,011 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23] comprehensivesort/(1/1) ...ive/false_true_false_true, VertexName: BFADFC004048489C9A7038D694BD14D8, VertexParallelism: 1, TaskAttemptID:attempt_1452786539755_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=20BA619F8EA943AC803713A6347D4878, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:00,011 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539755_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:00,011 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:00,012 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786539755_0001_00_000002
2016-01-14 15:49:00,012 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:00,014 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786539755_0001_00_000002
2016-01-14 15:49:00,014 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:49:00,014 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:00,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:00,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786539755_0001_00_000002, launchTime=1452786540015
2016-01-14 15:49:00,016 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786539755_0001_00_000002
2016-01-14 15:49:00,016 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786539755_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:49:00,017 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786539755_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:00,017 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786539755_0001_00_000001, Exiting
2016-01-14 15:49:00,017 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786539755_0001_00_000001
2016-01-14 15:49:00,017 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786539755_0001_00_000001
2016-01-14 15:49:00,017 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786539755_0001_1_01_000000_0] to container: [container_1452786539755_0001_00_000002]
2016-01-14 15:49:00,017 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:00,018 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786539755_0001_00_000002 given task: attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,019 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786539755_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,018 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786539755_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:00 Starting to run new task attempt: attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,018 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786539755_0001_1_01_000000_0] started. Is using containerId: [container_1452786539755_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:49:00,019 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:00,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BFADFC004048489C9A7038D694BD14D8, taskAttemptId=attempt_1452786539755_0001_1_01_000000_0, startTime=1452786540017, containerId=container_1452786539755_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786539755_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:49:00,020 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539755_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:00,020 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786539755_0001_00_000001 completed successfully
2016-01-14 15:49:00,020 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,020 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539755_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:00,020 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:49:00,020 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:00,020 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:00,021 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:00,021 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:00,021 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:00,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23] comprehensivesort/(1/1) ...ive/false_true_false_true, VertexName: BFADFC004048489C9A7038D694BD14D8, VertexParallelism: 1, TaskAttemptID:attempt_1452786539755_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=20BA619F8EA943AC803713A6347D4878, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:00,021 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:00,021 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:00,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,022 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786539755_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:49:00,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ABBD1A0EAF9E467C859C8AE9EAB31EF2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:49:00,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:00,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=20BA619F8EA943AC803713A6347D4878, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:49:00,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:00,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 20BA619F8EA943AC803713A6347D4878
2016-01-14 15:49:00,024 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:49:00,024 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 20BA619F8EA943AC803713A6347D4878
2016-01-14 15:49:00,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:49:00,025 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:49:00,026 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:49:00,027 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:00,031 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:49:00,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:49:00,033 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BFADFC004048489C9A7038D694BD14D8, ordinal: 1
2016-01-14 15:49:00,033 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BFADFC004048489C9A7038D694BD14D8, mem on start (mb), free: 201, total: 463, max: 672
2016-01-14 15:49:00,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:00,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:00,035 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:49:00,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 20BA619F8EA943AC803713A6347D4878
2016-01-14 15:49:00,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:00,036 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:00,036 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:49:00,036 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:00,037 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:49:00,037 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:00,037 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:49:00,038 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 20BA619F8EA943AC803713A6347D4878, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:49:00,038 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ABBD1A0EAF9E467C859C8AE9EAB31EF2, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:49:00,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:00,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ABBD1A0EAF9E467C859C8AE9EAB31EF2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:00,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:00,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:49:00,040 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:00,040 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:49:00,040 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ABBD1A0EAF9E467C859C8AE9EAB31EF2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:49:00,040 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:49:00,040 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:49:00,041 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:49:00,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:49:00,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:00,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:00,043 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,043 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786539755_0001_1_00_000000_0_10137, runDuration: 49441, ]
2016-01-14 15:49:00,044 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786539755_0001_1_00_000000_0_10137] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786540044, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 15:49:00,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:49:00,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:49:00,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [ABBD1A0EAF9E467C859C8AE9EAB31EF2] #24 in 1ms
2016-01-14 15:49:00,045 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ABBD1A0EAF9E467C859C8AE9EAB31EF2] #24, status:false, isInterrupted:false
2016-01-14 15:49:00,046 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:49:00,047 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:49:00,047 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:00,047 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:49:00,047 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:49:00,047 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:49:00,047 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:49:00,048 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:49:00,049 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 751FDB18FB6347718CAB35423E3E9404
2016-01-14 15:49:00,050 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true"], id: 20BA619F8EA943AC803713A6347D4878
2016-01-14 15:49:00,051 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true"]
2016-01-14 15:49:00,051 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 751FDB18FB6347718CAB35423E3E9404, for 1 inputs
2016-01-14 15:49:00,051 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BFADFC004048489C9A7038D694BD14D8, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:49:00,052 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@130ac6d3; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@28835069
2016-01-14 15:49:00,056 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:49:00,057 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:49:00,057 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,057 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786539755_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:49:00,058 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,058 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BFADFC004048489C9A7038D694BD14D8, mem on close (mb), free: 199, total: 463, max: 672
2016-01-14 15:49:00,058 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ABBD1A0EAF9E467C859C8AE9EAB31EF2
2016-01-14 15:49:00,059 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:49:00,059 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:49:00,059 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:00,060 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:49:00,060 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:49:00 Completed running task attempt: attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,061 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786539755_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:00,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=486014976, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786540052, Process_Duration=6, Process_End_Time=1452786540058, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:49:00,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BFADFC004048489C9A7038D694BD14D8, taskAttemptId=attempt_1452786539755_0001_1_01_000000_0, startTime=1452786540017, finishTime=1452786540062, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=486014976, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540052, Process_Duration=6, Process_End_Time=1452786540058, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:00,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:00,063 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:00,063 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,062 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786539755_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:00,063 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786539755_0001_1_01_000000_0
2016-01-14 15:49:00,063 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786539755_0001_00_000002
2016-01-14 15:49:00,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:00,064 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:00,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:TASK_FINISHED]: vertexName=BFADFC004048489C9A7038D694BD14D8, taskId=task_1452786539755_0001_1_01_000000, startTime=1452786540017, finishTime=1452786540063, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452786539755_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=486014976, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540052, Process_Duration=6, Process_End_Time=1452786540058, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:00,064 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786539755_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:00,064 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8] : 1
2016-01-14 15:49:00,064 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:49:00,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:00,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:VERTEX_FINISHED]: vertexName=BFADFC004048489C9A7038D694BD14D8, vertexId=vertex_1452786539755_0001_1_01, initRequestedTime=1452786539919, initedTime=1452786539929, startRequestedTime=1452786539935, startedTime=1452786539935, finishTime=1452786540064, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=486014976, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540052, Process_Duration=6, Process_End_Time=1452786540058, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786540017, firstTasksToStart=[ task_1452786539755_0001_1_01_000000 ], lastTaskFinishTime=1452786540062, lastTasksToFinish=[ task_1452786539755_0001_1_01_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786539755_0001_1_01_000000 ], longestDurationTasks=[ task_1452786539755_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:00,065 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:00,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:00,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786539755_0001_00_000002, stoppedTime=1452786540066, exitStatus=0
2016-01-14 15:49:00,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:00,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:00,066 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:00,066 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786539755_0001_1
2016-01-14 15:49:00,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786539755_0001_1
2016-01-14 15:49:00,066 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786539755_0001_1_00 [ABBD1A0EAF9E467C859C8AE9EAB31EF2]
2016-01-14 15:49:00,066 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 20BA619F8EA943AC803713A6347D4878 for vertex: vertex_1452786539755_0001_1_01 [BFADFC004048489C9A7038D694BD14D8]
2016-01-14 15:49:00,074 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786539755_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:49:00,074 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786539755_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:00,074 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786539755_0001_00_000002, Exiting
2016-01-14 15:49:00,074 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786539755_0001_00_000002
2016-01-14 15:49:00,074 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786539755_0001_00_000002
2016-01-14 15:49:00,074 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786539755_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:00,074 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786539755_0001_00_000002 completed successfully
2016-01-14 15:49:00,075 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786539755_0001_1, queueSize=0
2016-01-14 15:49:00,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:49:00,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786539755_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786539755_0001_1, startTime=1452786539916, finishTime=1452786540072, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=972029952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573080045, Process_Duration=11, Process_End_Time=2905573080056, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:49:00,076 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786539755_0001_1 finished with state: SUCCEEDED
2016-01-14 15:49:00,076 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786539755_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:49:00,076 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786539755_0001_00_000002
2016-01-14 15:49:00,077 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786539755_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:49:00 Completed Dag: dag_1452786539755_0001_1
2016-01-14 15:49:00,077 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786539755_0001_1, dagState=SUCCEEDED
2016-01-14 15:49:00,078 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786539755_0001_1_post
2016-01-14 15:49:00,078 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,078 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:49:00,080 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_true_false_true, applicationId=application_1452786539755_0001
2016-01-14 15:49:00,082 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:49:00,082 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:49:00,082 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:49:00,082 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:49:00,082 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:49:00,083 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:49:00,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:49:00,083 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:49:00,084 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:49:00,084 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:49:00,084 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:49:00,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:49:00,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:49:00,090 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0F8A537F4F98455E977FAE2648CD456F/BB439A73D8D0419E9A994575B0DCEC23/.tez/application_1452786539755_0001
2016-01-14 15:49:00,090 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.356
2016-01-14 15:49:00,140 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:49:00,140 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:49:00,171 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 15:49:00,174 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 15:49:00,175 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:49:00,175 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:49:00,175 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:49:00,187 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:49:00,196 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true
2016-01-14 15:49:00,235 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:49:00,235 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:00,235 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true"]
2016-01-14 15:49:00,235 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:49:00,236 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:49:00,236 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:49:00,237 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/false_false_true_true
2016-01-14 15:49:00,237 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:49:00,245 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:49:00,250 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:49:00,251 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:49:00,251 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:49:00,261 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244/.tez/application_1452786540251_0001 doesn't exist and is created
2016-01-14 15:49:00,277 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:49:00,277 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:49:00,278 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244/.tez/application_1452786540251_0001
2016-01-14 15:49:00,280 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786540251_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:49:00,288 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:49:00,288 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:49:00,289 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:49:00,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:49:00,290 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244/.tez/application_1452786540251_0001/localmode-log-dir
2016-01-14 15:49:00,290 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244/.tez/application_1452786540251_0001/localmode-log-dir/history.txt.appattempt_1452786540251_0001_000000, maxErrors=10
2016-01-14 15:49:00,290 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:49:00,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786540251_0001_000000, appSubmitTime=1452786540280, launchTime=1452786540280
2016-01-14 15:49:00,295 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:33898
2016-01-14 15:49:00,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:49:00,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:49:00,301 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:49:00,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:49:00,302 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:49:00,304 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786540251_0001_000000, startTime=1452786540303
2016-01-14 15:49:00,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:49:00,304 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:49:00,378 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:49:00,378 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:49:00,378 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_false_true_true, applicationId=application_1452786540251_0001, dagName=[8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244] comprehensivesort/(1/1) ...ive/false_false_true_true
2016-01-14 15:49:00,390 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244] comprehensivesort/(1/1) ...ive/false_false_true_true
2016-01-14 15:49:00,391 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786540251_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244/.tez/application_1452786540251_0001/localmode-log-dir/dag_1452786540251_0001_1.dot
2016-01-14 15:49:00,392 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244/.tez/application_1452786540251_0001/localmode-log-dir/dag_1452786540251_0001_1-tez-dag.pb.txt
2016-01-14 15:49:00,398 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786540251_0001_1
2016-01-14 15:49:00,399 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,399 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244] comprehensivesort/(1/1) ...ive/false_false_true_true
2016-01-14 15:49:00 Running Dag: dag_1452786540251_0001_1
2016-01-14 15:49:00,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786540251_0001_1, submitTime=1452786540390
2016-01-14 15:49:00,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:49:00,411 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D38F1DD82ACB45ADA3C45F91BCF91504
2016-01-14 15:49:00,412 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:49:00,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786540251_0001_1, initTime=1452786540411
2016-01-14 15:49:00,412 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:49:00,413 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786540251_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:49:00,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:DAG_STARTED]: dagID=dag_1452786540251_0001_1, startTime=1452786540413
2016-01-14 15:49:00,413 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:49:00,413 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786540251_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:49:00,413 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C702F14167AA47FDBFB39A6CC69707C7 : {CE3C0A963A18429C95963B06ABC82A5A={InputName=CE3C0A963A18429C95963B06ABC82A5A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:49:00,413 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: CE3C0A963A18429C95963B06ABC82A5A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:49:00,413 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,414 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,414 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:49:00,414 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]. Starting root input initializers: 1
2016-01-14 15:49:00,414 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:49:00,414 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: CE3C0A963A18429C95963B06ABC82A5A on vertex vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,414 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,415 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_false_true_true, applicationId=application_1452786540251_0001, dagName=[8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244] comprehensivesort/(1/1) ...ive/false_false_true_true
2016-01-14 15:49:00,417 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:49:00,418 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,418 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,418 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,418 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=22F66DDB78FB48F497DB205783966076, vertexId=vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:49:00,418 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=22F66DDB78FB48F497DB205783966076, vertexId=vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,419 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:49:00,419 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input CE3C0A963A18429C95963B06ABC82A5A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:49:00,420 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:49:00,420 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:49:00,421 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=22F66DDB78FB48F497DB205783966076, vertexName=D38F1DD82ACB45ADA3C45F91BCF91504, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:49:00,421 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=22F66DDB78FB48F497DB205783966076, vertexId=vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:49:00,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D38F1DD82ACB45ADA3C45F91BCF91504, vertexId=vertex_1452786540251_0001_1_01, initRequestedTime=1452786540414, initedTime=1452786540422, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:49:00,422 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:49:00,423 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:49:00,423 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: CE3C0A963A18429C95963B06ABC82A5A on vertex vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,423 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:49:00,424 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {CE3C0A963A18429C95963B06ABC82A5A=forAllWorkUnits=true, update=[1]}
2016-01-14 15:49:00,424 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] parallelism set to 1
2016-01-14 15:49:00,424 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:49:00,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786540251_0001_1_00, eventCount=1
2016-01-14 15:49:00,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C702F14167AA47FDBFB39A6CC69707C7, vertexId=vertex_1452786540251_0001_1_00, initRequestedTime=1452786540413, initedTime=1452786540425, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:49:00,425 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] with distanceFromRoot: 0
2016-01-14 15:49:00,425 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:49:00,425 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,426 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:49:00,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786540251_0001_1_00, startRequestedTime=1452786540425, startedTime=1452786540425
2016-01-14 15:49:00,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:49:00,426 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:49:00,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:00,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:TASK_STARTED]: vertexName=C702F14167AA47FDBFB39A6CC69707C7, taskId=task_1452786540251_0001_1_00_000000, scheduledTime=1452786540427, launchTime=1452786540427
2016-01-14 15:49:00,427 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540251_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:00,427 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786540251_0001_1_00 for vertex: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504] numStartedSources: 1 numSources: 1
2016-01-14 15:49:00,428 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504] with distanceFromRoot: 1
2016-01-14 15:49:00,428 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,428 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C702F14167AA47FDBFB39A6CC69707C7 in vertex: D38F1DD82ACB45ADA3C45F91BCF91504
2016-01-14 15:49:00,429 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D38F1DD82ACB45ADA3C45F91BCF91504 with 1 source tasks and 1 pending tasks
2016-01-14 15:49:00,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:49:00,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786540251_0001_1_01, startRequestedTime=1452786540428, startedTime=1452786540428
2016-01-14 15:49:00,429 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786540251_0001, with dag id: dag_1452786540251_0001_1
2016-01-14 15:49:00,429 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:49:00,430 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244] comprehensivesort/(1/1) ...ive/false_false_true_true, VertexName: C702F14167AA47FDBFB39A6CC69707C7, VertexParallelism: 1, TaskAttemptID:attempt_1452786540251_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE3C0A963A18429C95963B06ABC82A5A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D38F1DD82ACB45ADA3C45F91BCF91504, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:49:00,430 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540251_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:00,430 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:00,431 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:49:00,432 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:49:00,432 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:49:00,432 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786540251_0001_00_000001
2016-01-14 15:49:00,432 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:00,433 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786540251_0001_00_000001
2016-01-14 15:49:00,433 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:49:00,434 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:00,434 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:00,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786540251_0001_00_000001, launchTime=1452786540434
2016-01-14 15:49:00,435 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786540251_0001_00_000001
2016-01-14 15:49:00,435 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786540251_0001_1_00_000000_0] to container: [container_1452786540251_0001_00_000001]
2016-01-14 15:49:00,435 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:00,436 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786540251_0001_1_00_000000_0] started. Is using containerId: [container_1452786540251_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:49:00,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C702F14167AA47FDBFB39A6CC69707C7, taskAttemptId=attempt_1452786540251_0001_1_00_000000_0, startTime=1452786540436, containerId=container_1452786540251_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786540251_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:49:00,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:00,436 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540251_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:00,436 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786540251_0001_00_000001 given task: attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00,436 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540251_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:00,437 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786540251_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00 Starting to run new task attempt: attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00,437 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,438 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:00,438 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:00,438 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:00,438 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:00,438 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:00,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244] comprehensivesort/(1/1) ...ive/false_false_true_true, VertexName: C702F14167AA47FDBFB39A6CC69707C7, VertexParallelism: 1, TaskAttemptID:attempt_1452786540251_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE3C0A963A18429C95963B06ABC82A5A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D38F1DD82ACB45ADA3C45F91BCF91504, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:49:00,438 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:00,439 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:00,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,439 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786540251_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:49:00,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D38F1DD82ACB45ADA3C45F91BCF91504, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:49:00,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CE3C0A963A18429C95963B06ABC82A5A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:49:00,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:00,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:00,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CE3C0A963A18429C95963B06ABC82A5A
2016-01-14 15:49:00,441 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: CE3C0A963A18429C95963B06ABC82A5A
2016-01-14 15:49:00,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D38F1DD82ACB45ADA3C45F91BCF91504
2016-01-14 15:49:00,443 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:49:00,443 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: CE3C0A963A18429C95963B06ABC82A5A
2016-01-14 15:49:00,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CE3C0A963A18429C95963B06ABC82A5A
2016-01-14 15:49:00,443 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:49:00,443 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:00,443 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D38F1DD82ACB45ADA3C45F91BCF91504
2016-01-14 15:49:00,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D38F1DD82ACB45ADA3C45F91BCF91504
2016-01-14 15:49:00,449 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C702F14167AA47FDBFB39A6CC69707C7, ordinal: 0
2016-01-14 15:49:00,449 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C702F14167AA47FDBFB39A6CC69707C7, mem on start (mb), free: 117, total: 463, max: 672
2016-01-14 15:49:00,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:00,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:00,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:00,450 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:00,450 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:49:00,450 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:00,450 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:49:00,451 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:00,451 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:49:00,451 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CE3C0A963A18429C95963B06ABC82A5A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:49:00,451 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D38F1DD82ACB45ADA3C45F91BCF91504, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:49:00,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:00,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CE3C0A963A18429C95963B06ABC82A5A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:00,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CE3C0A963A18429C95963B06ABC82A5A
2016-01-14 15:49:00,452 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:49:00,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CE3C0A963A18429C95963B06ABC82A5A
2016-01-14 15:49:00,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:00,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:00,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:00,453 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00,453 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:49:00,454 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2e53cb0f
2016-01-14 15:49:00,455 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2e53cb0f
2016-01-14 15:49:00,455 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:49:00,455 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:49:00,456 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:00,457 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: CE3C0A963A18429C95963B06ABC82A5A
2016-01-14 15:49:00,457 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: F7EA46212B104682AF9A25A426A8C6DC
2016-01-14 15:49:00,457 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] F7EA46212B104682AF9A25A426A8C6DC
2016-01-14 15:49:00,458 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@56be1bb1; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5b0c2df4; comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@7c9d1cde; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:49:00,458 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:49:00,458 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:49:00,458 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:00,479 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:49:00,479 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:49:00,480 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:49:00,480 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:49:00,480 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:49:00,482 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:00,482 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:49:00,482 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C702F14167AA47FDBFB39A6CC69707C7, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:49:00,486 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00,486 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C702F14167AA47FDBFB39A6CC69707C7, mem on close (mb), free: 215, total: 473, max: 672
2016-01-14 15:49:00,487 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:49:00,487 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:49:00,487 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:49:00,487 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:49:00,491 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:49:00 Completed running task attempt: attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00,494 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786540251_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:00,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786540482, Process_Duration=4, Process_End_Time=1452786540486, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:49:00,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786540251_0001_1_00, eventCount=1
2016-01-14 15:49:00,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C702F14167AA47FDBFB39A6CC69707C7, taskAttemptId=attempt_1452786540251_0001_1_00_000000_0, startTime=1452786540436, finishTime=1452786540495, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540482, Process_Duration=4, Process_End_Time=1452786540486, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:00,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:00,495 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:00,495 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,495 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540251_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:00,496 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786540251_0001_00_000001
2016-01-14 15:49:00,496 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:00,496 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786540251_0001_1_00_000000_0
2016-01-14 15:49:00,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:00,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:TASK_FINISHED]: vertexName=C702F14167AA47FDBFB39A6CC69707C7, taskId=task_1452786540251_0001_1_00_000000, startTime=1452786540436, finishTime=1452786540496, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1452786540251_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540482, Process_Duration=4, Process_End_Time=1452786540486, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:00,497 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540251_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:00,497 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] : 1
2016-01-14 15:49:00,497 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:00,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:VERTEX_FINISHED]: vertexName=C702F14167AA47FDBFB39A6CC69707C7, vertexId=vertex_1452786540251_0001_1_00, initRequestedTime=1452786540413, initedTime=1452786540425, startRequestedTime=1452786540425, startedTime=1452786540425, finishTime=1452786540497, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540482, Process_Duration=4, Process_End_Time=1452786540486, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786540436, firstTasksToStart=[ task_1452786540251_0001_1_00_000000 ], lastTaskFinishTime=1452786540495, lastTasksToFinish=[ task_1452786540251_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786540251_0001_1_00_000000 ], longestDurationTasks=[ task_1452786540251_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:00,498 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:00,498 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:00,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786540251_0001_00_000001, stoppedTime=1452786540498, exitStatus=0
2016-01-14 15:49:00,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:00,499 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:00,499 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504] attempt: attempt_1452786540251_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:49:00,499 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D38F1DD82ACB45ADA3C45F91BCF91504
2016-01-14 15:49:00,499 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,499 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:00,499 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786540251_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:49:00,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:00,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:TASK_STARTED]: vertexName=D38F1DD82ACB45ADA3C45F91BCF91504, taskId=task_1452786540251_0001_1_01_000000, scheduledTime=1452786540499, launchTime=1452786540499
2016-01-14 15:49:00,500 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540251_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:00,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:49:00,500 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,500 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244] comprehensivesort/(1/1) ...ive/false_false_true_true, VertexName: D38F1DD82ACB45ADA3C45F91BCF91504, VertexParallelism: 1, TaskAttemptID:attempt_1452786540251_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C702F14167AA47FDBFB39A6CC69707C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=22F66DDB78FB48F497DB205783966076, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:00,500 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540251_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:00,501 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:00,502 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786540251_0001_00_000002
2016-01-14 15:49:00,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:00,504 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786540251_0001_00_000002
2016-01-14 15:49:00,504 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:49:00,505 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:00,505 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:00,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786540251_0001_00_000002, launchTime=1452786540505
2016-01-14 15:49:00,506 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786540251_0001_00_000002
2016-01-14 15:49:00,506 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786540251_0001_1_01_000000_0] to container: [container_1452786540251_0001_00_000002]
2016-01-14 15:49:00,506 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786540251_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:49:00,506 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786540251_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:00,506 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786540251_0001_00_000001, Exiting
2016-01-14 15:49:00,507 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786540251_0001_00_000001
2016-01-14 15:49:00,507 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786540251_0001_00_000001
2016-01-14 15:49:00,506 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:00,507 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786540251_0001_1_01_000000_0] started. Is using containerId: [container_1452786540251_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:49:00,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D38F1DD82ACB45ADA3C45F91BCF91504, taskAttemptId=attempt_1452786540251_0001_1_01_000000_0, startTime=1452786540507, containerId=container_1452786540251_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786540251_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:49:00,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:00,508 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540251_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:00,508 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540251_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:00,508 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786540251_0001_00_000002 given task: attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,508 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786540251_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00 Starting to run new task attempt: attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,509 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,509 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,509 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:00,509 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:00,509 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:00,509 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:00,509 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:00,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244] comprehensivesort/(1/1) ...ive/false_false_true_true, VertexName: D38F1DD82ACB45ADA3C45F91BCF91504, VertexParallelism: 1, TaskAttemptID:attempt_1452786540251_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C702F14167AA47FDBFB39A6CC69707C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=22F66DDB78FB48F497DB205783966076, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:00,510 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:00,510 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786540251_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:00,510 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786540251_0001_00_000001 completed successfully
2016-01-14 15:49:00,510 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:49:00,511 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:00,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,511 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786540251_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:49:00,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C702F14167AA47FDBFB39A6CC69707C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:49:00,513 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:00,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=22F66DDB78FB48F497DB205783966076, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:49:00,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:00,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 22F66DDB78FB48F497DB205783966076
2016-01-14 15:49:00,515 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:49:00,515 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 22F66DDB78FB48F497DB205783966076
2016-01-14 15:49:00,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:00,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,517 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:49:00,518 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:49:00,518 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,521 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D38F1DD82ACB45ADA3C45F91BCF91504, ordinal: 1
2016-01-14 15:49:00,521 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D38F1DD82ACB45ADA3C45F91BCF91504, mem on start (mb), free: 209, total: 473, max: 672
2016-01-14 15:49:00,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:00,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:00,525 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:49:00,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 22F66DDB78FB48F497DB205783966076
2016-01-14 15:49:00,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:00,526 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:00,526 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:49:00,526 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:00,526 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:49:00,526 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:00,526 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:49:00,527 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 22F66DDB78FB48F497DB205783966076, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:49:00,527 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C702F14167AA47FDBFB39A6CC69707C7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:49:00,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:00,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C702F14167AA47FDBFB39A6CC69707C7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:00,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:00,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:00,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,529 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:00,529 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:49:00,530 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C702F14167AA47FDBFB39A6CC69707C7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:49:00,530 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:49:00,530 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:49:00,530 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:49:00,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:00,532 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,536 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:49:00,536 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: F7EA46212B104682AF9A25A426A8C6DC
2016-01-14 15:49:00,536 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true"], id: 22F66DDB78FB48F497DB205783966076
2016-01-14 15:49:00,537 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true"]
2016-01-14 15:49:00,537 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] F7EA46212B104682AF9A25A426A8C6DC, for 1 inputs
2016-01-14 15:49:00,537 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786540251_0001_1_00_000000_0_10142, runDuration: 51278, ]
2016-01-14 15:49:00,539 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786540251_0001_1_00_000000_0_10142] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786540539, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:49:00,540 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,540 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:49:00,540 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C702F14167AA47FDBFB39A6CC69707C7] #25 in 2ms
2016-01-14 15:49:00,540 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C702F14167AA47FDBFB39A6CC69707C7] #25, status:false, isInterrupted:false
2016-01-14 15:49:00,541 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:49:00,541 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:49:00,541 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:00,542 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:49:00,542 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:49:00,542 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,542 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:49:00,543 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D38F1DD82ACB45ADA3C45F91BCF91504, all 1 inputs ready in: 00:00:00.005
2016-01-14 15:49:00,543 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@68cd9e28; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@134a89d9
2016-01-14 15:49:00,548 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:49:00,548 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:49:00,549 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,549 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786540251_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:49:00,551 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,551 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D38F1DD82ACB45ADA3C45F91BCF91504, mem on close (mb), free: 207, total: 473, max: 672
2016-01-14 15:49:00,551 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C702F14167AA47FDBFB39A6CC69707C7
2016-01-14 15:49:00,551 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:49:00,552 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:49:00,552 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:00,552 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:49:00,552 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:49:00 Completed running task attempt: attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,553 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786540251_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:00,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:00,554 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D38F1DD82ACB45ADA3C45F91BCF91504, taskAttemptId=attempt_1452786540251_0001_1_01_000000_0, startTime=1452786540507, finishTime=1452786540554, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540543, Process_Duration=8, Process_End_Time=1452786540551, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:00,554 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540251_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:00,555 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:00,555 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786540251_0001_1_01_000000_0
2016-01-14 15:49:00,555 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:00,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:TASK_FINISHED]: vertexName=D38F1DD82ACB45ADA3C45F91BCF91504, taskId=task_1452786540251_0001_1_01_000000, startTime=1452786540507, finishTime=1452786540555, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1452786540251_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540543, Process_Duration=8, Process_End_Time=1452786540551, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:00,555 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540251_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:00,556 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504] : 1
2016-01-14 15:49:00,556 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,556 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:00,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:VERTEX_FINISHED]: vertexName=D38F1DD82ACB45ADA3C45F91BCF91504, vertexId=vertex_1452786540251_0001_1_01, initRequestedTime=1452786540414, initedTime=1452786540422, startRequestedTime=1452786540428, startedTime=1452786540428, finishTime=1452786540556, timeTaken=128, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540543, Process_Duration=8, Process_End_Time=1452786540551, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786540507, firstTasksToStart=[ task_1452786540251_0001_1_01_000000 ], lastTaskFinishTime=1452786540554, lastTasksToFinish=[ task_1452786540251_0001_1_01_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786540251_0001_1_01_000000 ], longestDurationTasks=[ task_1452786540251_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:00,557 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:00,557 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:00,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786540251_0001_00_000002, stoppedTime=1452786540557, exitStatus=0
2016-01-14 15:49:00,558 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:00,558 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:00,558 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786540251_0001_1
2016-01-14 15:49:00,558 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786540251_0001_1
2016-01-14 15:49:00,558 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 22F66DDB78FB48F497DB205783966076 for vertex: vertex_1452786540251_0001_1_01 [D38F1DD82ACB45ADA3C45F91BCF91504]
2016-01-14 15:49:00,558 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:00,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786540543, Process_Duration=8, Process_End_Time=1452786540551, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:49:00,564 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:00,564 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,564 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786540251_0001_00_000002
2016-01-14 15:49:00,564 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786540251_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:49:00,564 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786540251_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:00,565 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786540251_0001_00_000002, Exiting
2016-01-14 15:49:00,565 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786540251_0001_00_000002
2016-01-14 15:49:00,565 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786540251_0001_00_000002
2016-01-14 15:49:00,565 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786540251_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:00,565 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786540251_0001_00_000002 completed successfully
2016-01-14 15:49:00,570 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786540251_0001_1_00 [C702F14167AA47FDBFB39A6CC69707C7]
2016-01-14 15:49:00,572 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786540251_0001_1, queueSize=0
2016-01-14 15:49:00,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:49:00,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540251_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786540251_0001_1, startTime=1452786540413, finishTime=1452786540570, timeTaken=157, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=993001472, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573081025, Process_Duration=12, Process_End_Time=2905573081037, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:49:00,572 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786540251_0001_1 finished with state: SUCCEEDED
2016-01-14 15:49:00,573 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786540251_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:49:00,573 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540251_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:49:00,573 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786540251_0001_00_000002
2016-01-14 15:49:00 Completed Dag: dag_1452786540251_0001_1
2016-01-14 15:49:00,575 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786540251_0001_1, dagState=SUCCEEDED
2016-01-14 15:49:00,575 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786540251_0001_1_post
2016-01-14 15:49:00,576 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,576 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:49:00,578 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_false_true_true, applicationId=application_1452786540251_0001
2016-01-14 15:49:00,579 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:49:00,579 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:49:00,579 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:49:00,580 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:49:00,580 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:49:00,580 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:49:00,580 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:49:00,580 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:49:00,580 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:49:00,580 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:49:00,581 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:49:00,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:49:00,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:49:00,585 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CC8EA1EF50B4BFD9E5B39B0F1C34B28/FFB76A5CF4DE422F9AFFEF1F7DC58244/.tez/application_1452786540251_0001
2016-01-14 15:49:00,586 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.350
2016-01-14 15:49:00,651 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:49:00,651 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:49:00,680 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 15:49:00,698 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 15:49:00,699 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:49:00,699 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:49:00,699 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:49:00,709 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:49:00,715 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true
2016-01-14 15:49:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:49:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true"]
2016-01-14 15:49:00,749 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:49:00,749 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:49:00,749 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:49:00,749 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...sive/true_false_true_true
2016-01-14 15:49:00,750 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:49:00,758 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:49:00,762 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:49:00,763 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:49:00,763 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:49:00,772 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804/.tez/application_1452786540762_0001 doesn't exist and is created
2016-01-14 15:49:00,786 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:49:00,786 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:49:00,786 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804/.tez/application_1452786540762_0001
2016-01-14 15:49:00,789 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786540762_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:49:00,797 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:49:00,797 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:49:00,798 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:49:00,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:49:00,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804/.tez/application_1452786540762_0001/localmode-log-dir
2016-01-14 15:49:00,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804/.tez/application_1452786540762_0001/localmode-log-dir/history.txt.appattempt_1452786540762_0001_000000, maxErrors=10
2016-01-14 15:49:00,799 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:49:00,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786540762_0001_000000, appSubmitTime=1452786540789, launchTime=1452786540789
2016-01-14 15:49:00,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:49:00,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:49:00,805 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:43735
2016-01-14 15:49:00,807 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:49:00,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:49:00,808 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:49:00,810 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786540762_0001_000000, startTime=1452786540810
2016-01-14 15:49:00,810 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:49:00,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:49:00,886 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:49:00,886 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:49:00,886 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_false_true_true, applicationId=application_1452786540762_0001, dagName=[16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804] comprehensivesort/(1/1) ...sive/true_false_true_true
2016-01-14 15:49:00,898 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804] comprehensivesort/(1/1) ...sive/true_false_true_true
2016-01-14 15:49:00,899 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786540762_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804/.tez/application_1452786540762_0001/localmode-log-dir/dag_1452786540762_0001_1.dot
2016-01-14 15:49:00,900 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804/.tez/application_1452786540762_0001/localmode-log-dir/dag_1452786540762_0001_1-tez-dag.pb.txt
2016-01-14 15:49:00,907 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786540762_0001_1
2016-01-14 15:49:00,907 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,907 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804] comprehensivesort/(1/1) ...sive/true_false_true_true
2016-01-14 15:49:00 Running Dag: dag_1452786540762_0001_1
2016-01-14 15:49:00,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786540762_0001_1, submitTime=1452786540898
2016-01-14 15:49:00,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:49:00,919 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CE701424E12E45B29BC188054EB160E3
2016-01-14 15:49:00,919 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:49:00,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786540762_0001_1, initTime=1452786540918
2016-01-14 15:49:00,919 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786540762_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:49:00,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:49:00,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:DAG_STARTED]: dagID=dag_1452786540762_0001_1, startTime=1452786540919
2016-01-14 15:49:00,920 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:49:00,920 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_false_true_true, applicationId=application_1452786540762_0001, dagName=[16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804] comprehensivesort/(1/1) ...sive/true_false_true_true
2016-01-14 15:49:00,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:49:00,920 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786540762_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:49:00,921 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FA377A46CEE84747BD0F6CBB8377AAE6 : {01CD059436A942B1B918B73CFCCCD684={InputName=01CD059436A942B1B918B73CFCCCD684}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:49:00,921 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 01CD059436A942B1B918B73CFCCCD684, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:49:00,921 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:00,922 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:00,922 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:00,922 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]. Starting root input initializers: 1
2016-01-14 15:49:00,922 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:49:00,922 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:00,924 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 01CD059436A942B1B918B73CFCCCD684 on vertex vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:00,926 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:49:00,926 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 01CD059436A942B1B918B73CFCCCD684 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:49:00,926 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:49:00,927 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:00,927 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:00,927 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:00,927 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CD5592991BCD48A983552E23A4C6D9FC, vertexId=vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:49:00,928 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CD5592991BCD48A983552E23A4C6D9FC, vertexId=vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:00,927 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:49:00,928 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:49:00,930 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CD5592991BCD48A983552E23A4C6D9FC, vertexName=CE701424E12E45B29BC188054EB160E3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:49:00,930 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CD5592991BCD48A983552E23A4C6D9FC, vertexId=vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:00,930 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:49:00,931 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 01CD059436A942B1B918B73CFCCCD684 on vertex vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:00,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:49:00,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CE701424E12E45B29BC188054EB160E3, vertexId=vertex_1452786540762_0001_1_01, initRequestedTime=1452786540922, initedTime=1452786540931, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:49:00,932 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:49:00,932 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:49:00,932 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {01CD059436A942B1B918B73CFCCCD684=forAllWorkUnits=true, update=[1]}
2016-01-14 15:49:00,932 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] parallelism set to 1
2016-01-14 15:49:00,932 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:49:00,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786540762_0001_1_00, eventCount=1
2016-01-14 15:49:00,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:49:00,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA377A46CEE84747BD0F6CBB8377AAE6, vertexId=vertex_1452786540762_0001_1_00, initRequestedTime=1452786540921, initedTime=1452786540933, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:49:00,933 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] with distanceFromRoot: 0
2016-01-14 15:49:00,934 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:49:00,934 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786540762_0001, with dag id: dag_1452786540762_0001_1
2016-01-14 15:49:00,934 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:00,934 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:00,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:49:00,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786540762_0001_1_00, startRequestedTime=1452786540934, startedTime=1452786540934
2016-01-14 15:49:00,935 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:49:00,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:00,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:TASK_STARTED]: vertexName=FA377A46CEE84747BD0F6CBB8377AAE6, taskId=task_1452786540762_0001_1_00_000000, scheduledTime=1452786540935, launchTime=1452786540935
2016-01-14 15:49:00,936 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540762_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:00,936 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786540762_0001_1_00 for vertex: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3] numStartedSources: 1 numSources: 1
2016-01-14 15:49:00,936 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3] with distanceFromRoot: 1
2016-01-14 15:49:00,937 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:00,937 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FA377A46CEE84747BD0F6CBB8377AAE6 in vertex: CE701424E12E45B29BC188054EB160E3
2016-01-14 15:49:00,937 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CE701424E12E45B29BC188054EB160E3 with 1 source tasks and 1 pending tasks
2016-01-14 15:49:00,937 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:49:00,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786540762_0001_1_01, startRequestedTime=1452786540937, startedTime=1452786540937
2016-01-14 15:49:00,938 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:49:00,938 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804] comprehensivesort/(1/1) ...sive/true_false_true_true, VertexName: FA377A46CEE84747BD0F6CBB8377AAE6, VertexParallelism: 1, TaskAttemptID:attempt_1452786540762_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=01CD059436A942B1B918B73CFCCCD684, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE701424E12E45B29BC188054EB160E3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:49:00,938 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540762_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:00,938 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:00,939 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:49:00,940 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:49:00,940 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:49:00,941 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786540762_0001_00_000001
2016-01-14 15:49:00,941 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:00,942 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786540762_0001_00_000001
2016-01-14 15:49:00,942 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:49:00,943 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:00,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:00,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786540762_0001_00_000001, launchTime=1452786540943
2016-01-14 15:49:00,944 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786540762_0001_00_000001
2016-01-14 15:49:00,944 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786540762_0001_1_00_000000_0] to container: [container_1452786540762_0001_00_000001]
2016-01-14 15:49:00,944 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:00,945 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786540762_0001_00_000001 given task: attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:00,945 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786540762_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:00,945 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786540762_0001_1_00_000000_0] started. Is using containerId: [container_1452786540762_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:49:00,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA377A46CEE84747BD0F6CBB8377AAE6, taskAttemptId=attempt_1452786540762_0001_1_00_000000_0, startTime=1452786540945, containerId=container_1452786540762_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786540762_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:49:00,945 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540762_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:00,945 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540762_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:00 Starting to run new task attempt: attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:00,946 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:00,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:00,946 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:00,947 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:00,947 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:00,947 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:00,947 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:00,948 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:00,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804] comprehensivesort/(1/1) ...sive/true_false_true_true, VertexName: FA377A46CEE84747BD0F6CBB8377AAE6, VertexParallelism: 1, TaskAttemptID:attempt_1452786540762_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=01CD059436A942B1B918B73CFCCCD684, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE701424E12E45B29BC188054EB160E3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:49:00,948 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:00,948 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:00,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:00,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,949 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786540762_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:49:00,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=01CD059436A942B1B918B73CFCCCD684, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:49:00,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CE701424E12E45B29BC188054EB160E3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:49:00,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:00,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CE701424E12E45B29BC188054EB160E3
2016-01-14 15:49:00,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:00,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 01CD059436A942B1B918B73CFCCCD684
2016-01-14 15:49:00,951 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 01CD059436A942B1B918B73CFCCCD684
2016-01-14 15:49:00,953 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:00,955 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:49:00,955 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 01CD059436A942B1B918B73CFCCCD684
2016-01-14 15:49:00,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 01CD059436A942B1B918B73CFCCCD684
2016-01-14 15:49:00,953 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:49:00,955 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CE701424E12E45B29BC188054EB160E3
2016-01-14 15:49:00,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CE701424E12E45B29BC188054EB160E3
2016-01-14 15:49:00,958 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA377A46CEE84747BD0F6CBB8377AAE6, ordinal: 0
2016-01-14 15:49:00,958 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA377A46CEE84747BD0F6CBB8377AAE6, mem on start (mb), free: 123, total: 473, max: 672
2016-01-14 15:49:00,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:00,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:00,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:00,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:00,959 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:00,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:49:00,960 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:00,962 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:49:00,963 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:00,963 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:49:00,963 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 01CD059436A942B1B918B73CFCCCD684, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:49:00,963 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CE701424E12E45B29BC188054EB160E3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:49:00,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:00,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 01CD059436A942B1B918B73CFCCCD684 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:00,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:00,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:00,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 01CD059436A942B1B918B73CFCCCD684
2016-01-14 15:49:00,965 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:49:00,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 01CD059436A942B1B918B73CFCCCD684
2016-01-14 15:49:00,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:00,966 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:00,967 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:49:00,968 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@57237e51
2016-01-14 15:49:00,968 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:49:00,968 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:49:00,969 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:00,970 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 01CD059436A942B1B918B73CFCCCD684
2016-01-14 15:49:00,970 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: AE124D6D9B4A44C1896FB08A79AEE7DC
2016-01-14 15:49:00,970 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] AE124D6D9B4A44C1896FB08A79AEE7DC
2016-01-14 15:49:00,970 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@57237e51
2016-01-14 15:49:00,971 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:49:00,971 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:49:00,971 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@2444f28a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2a8f54a0; comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@27fcea34; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:49:00,971 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:49:00,972 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:49:00,972 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:00,993 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:49:00,993 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:49:00,993 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:49:00,996 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:49:00,996 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:49:00,997 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:00,998 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:49:00,998 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA377A46CEE84747BD0F6CBB8377AAE6, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:49:01,003 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:01,004 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA377A46CEE84747BD0F6CBB8377AAE6, mem on close (mb), free: 213, total: 473, max: 672
2016-01-14 15:49:01,005 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:49:01,005 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:49:01,005 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:49:01,005 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:49:01,008 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:49:01 Completed running task attempt: attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:01,011 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786540762_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:01,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786540762_0001_1_00, eventCount=1
2016-01-14 15:49:01,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:01,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA377A46CEE84747BD0F6CBB8377AAE6, taskAttemptId=attempt_1452786540762_0001_1_00_000000_0, startTime=1452786540945, finishTime=1452786541012, timeTaken=67, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540998, Process_Duration=5, Process_End_Time=1452786541003, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:01,012 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540762_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:01,012 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786540762_0001_1_00_000000_0
2016-01-14 15:49:01,012 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:01,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:TASK_FINISHED]: vertexName=FA377A46CEE84747BD0F6CBB8377AAE6, taskId=task_1452786540762_0001_1_00_000000, startTime=1452786540945, finishTime=1452786541012, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1452786540762_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540998, Process_Duration=5, Process_End_Time=1452786541003, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:01,012 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540762_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:01,014 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] : 1
2016-01-14 15:49:01,014 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:01,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:VERTEX_FINISHED]: vertexName=FA377A46CEE84747BD0F6CBB8377AAE6, vertexId=vertex_1452786540762_0001_1_00, initRequestedTime=1452786540921, initedTime=1452786540933, startRequestedTime=1452786540934, startedTime=1452786540934, finishTime=1452786541014, timeTaken=80, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786540998, Process_Duration=5, Process_End_Time=1452786541003, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786540945, firstTasksToStart=[ task_1452786540762_0001_1_00_000000 ], lastTaskFinishTime=1452786541012, lastTasksToFinish=[ task_1452786540762_0001_1_00_000000 ], minTaskDuration=67, maxTaskDuration=67, avgTaskDuration=67.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786540762_0001_1_00_000000 ], longestDurationTasks=[ task_1452786540762_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:01,014 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:01,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:01,015 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:01,015 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3] attempt: attempt_1452786540762_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:49:01,015 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CE701424E12E45B29BC188054EB160E3
2016-01-14 15:49:01,015 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:01,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786540762_0001_00_000001, stoppedTime=1452786541015, exitStatus=0
2016-01-14 15:49:01,016 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:01,016 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:01,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:01,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:TASK_STARTED]: vertexName=CE701424E12E45B29BC188054EB160E3, taskId=task_1452786540762_0001_1_01_000000, scheduledTime=1452786541016, launchTime=1452786541016
2016-01-14 15:49:01,016 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540762_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:01,016 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786540762_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:49:01,016 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:01,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:01,017 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:49:01,017 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:01,017 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804] comprehensivesort/(1/1) ...sive/true_false_true_true, VertexName: CE701424E12E45B29BC188054EB160E3, VertexParallelism: 1, TaskAttemptID:attempt_1452786540762_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FA377A46CEE84747BD0F6CBB8377AAE6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CD5592991BCD48A983552E23A4C6D9FC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:01,017 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540762_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:01,017 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:01,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786540998, Process_Duration=5, Process_End_Time=1452786541003, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:49:01,017 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786540762_0001_00_000002
2016-01-14 15:49:01,018 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:01,018 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:01,018 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,018 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786540762_0001_00_000001
2016-01-14 15:49:01,018 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786540762_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:49:01,018 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786540762_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:01,019 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786540762_0001_00_000001, Exiting
2016-01-14 15:49:01,019 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786540762_0001_00_000001
2016-01-14 15:49:01,019 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786540762_0001_00_000001
2016-01-14 15:49:01,019 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786540762_0001_00_000002
2016-01-14 15:49:01,020 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:49:01,021 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786540762_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:01,021 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786540762_0001_00_000001 completed successfully
2016-01-14 15:49:01,021 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:49:01,022 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:01,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:01,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786540762_0001_00_000002, launchTime=1452786541022
2016-01-14 15:49:01,023 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786540762_0001_00_000002
2016-01-14 15:49:01,023 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786540762_0001_1_01_000000_0] to container: [container_1452786540762_0001_00_000002]
2016-01-14 15:49:01,023 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:01,024 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786540762_0001_00_000002 given task: attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,024 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786540762_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,024 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786540762_0001_1_01_000000_0] started. Is using containerId: [container_1452786540762_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:49:01,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CE701424E12E45B29BC188054EB160E3, taskAttemptId=attempt_1452786540762_0001_1_01_000000_0, startTime=1452786541024, containerId=container_1452786540762_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786540762_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:49:01,024 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540762_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:01,024 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540762_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:01 Starting to run new task attempt: attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:01,025 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,025 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,025 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:01,025 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:01,025 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:01,025 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:01,025 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:01,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804] comprehensivesort/(1/1) ...sive/true_false_true_true, VertexName: CE701424E12E45B29BC188054EB160E3, VertexParallelism: 1, TaskAttemptID:attempt_1452786540762_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FA377A46CEE84747BD0F6CBB8377AAE6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CD5592991BCD48A983552E23A4C6D9FC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:01,026 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:01,026 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:01,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,027 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786540762_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:49:01,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CD5592991BCD48A983552E23A4C6D9FC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:49:01,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FA377A46CEE84747BD0F6CBB8377AAE6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:49:01,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:01,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CD5592991BCD48A983552E23A4C6D9FC
2016-01-14 15:49:01,028 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:49:01,028 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CD5592991BCD48A983552E23A4C6D9FC
2016-01-14 15:49:01,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:01,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:01,030 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:01,031 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:49:01,031 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:49:01,032 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:01,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:01,036 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CE701424E12E45B29BC188054EB160E3, ordinal: 1
2016-01-14 15:49:01,036 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE701424E12E45B29BC188054EB160E3, mem on start (mb), free: 208, total: 473, max: 672
2016-01-14 15:49:01,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:01,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:01,038 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:49:01,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CD5592991BCD48A983552E23A4C6D9FC
2016-01-14 15:49:01,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:01,039 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:01,039 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:49:01,039 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:01,039 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:49:01,039 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:01,039 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:49:01,039 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CD5592991BCD48A983552E23A4C6D9FC, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:49:01,040 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FA377A46CEE84747BD0F6CBB8377AAE6, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:49:01,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:01,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FA377A46CEE84747BD0F6CBB8377AAE6 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:01,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:01,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:01,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:01,041 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:01,041 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:49:01,041 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FA377A46CEE84747BD0F6CBB8377AAE6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:49:01,041 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:49:01,041 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:49:01,042 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:49:01,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:01,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:01,043 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,043 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786540762_0001_1_00_000000_0_10149, runDuration: 59299, ]
2016-01-14 15:49:01,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786540762_0001_1_00_000000_0_10149] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786541045, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:49:01,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:01,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:49:01,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FA377A46CEE84747BD0F6CBB8377AAE6] #26 in 1ms
2016-01-14 15:49:01,045 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:49:01,045 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FA377A46CEE84747BD0F6CBB8377AAE6] #26, status:false, isInterrupted:false
2016-01-14 15:49:01,046 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:49:01,046 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: AE124D6D9B4A44C1896FB08A79AEE7DC
2016-01-14 15:49:01,046 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true"], id: CD5592991BCD48A983552E23A4C6D9FC
2016-01-14 15:49:01,046 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true"]
2016-01-14 15:49:01,046 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:49:01,046 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] AE124D6D9B4A44C1896FB08A79AEE7DC, for 1 inputs
2016-01-14 15:49:01,047 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:01,047 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:49:01,047 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:49:01,047 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:01,047 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:49:01,048 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CE701424E12E45B29BC188054EB160E3, all 1 inputs ready in: 00:00:00.001
2016-01-14 15:49:01,048 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@285eae2e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5ffd62b6
2016-01-14 15:49:01,054 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:49:01,054 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:49:01,054 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,054 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786540762_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:49:01,055 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,055 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE701424E12E45B29BC188054EB160E3, mem on close (mb), free: 205, total: 473, max: 672
2016-01-14 15:49:01,055 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FA377A46CEE84747BD0F6CBB8377AAE6
2016-01-14 15:49:01,060 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:49:01,061 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:49:01,061 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:01,061 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:49:01,061 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:49:01 Completed running task attempt: attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,062 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786540762_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:01,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786541048, Process_Duration=7, Process_End_Time=1452786541055, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:49:01,063 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:01,064 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:01,064 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,064 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786540762_0001_00_000002
2016-01-14 15:49:01,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CE701424E12E45B29BC188054EB160E3, taskAttemptId=attempt_1452786540762_0001_1_01_000000_0, startTime=1452786541024, finishTime=1452786541063, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541048, Process_Duration=7, Process_End_Time=1452786541055, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:01,064 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786540762_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:01,064 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:01,064 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786540762_0001_1_01_000000_0
2016-01-14 15:49:01,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:01,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:TASK_FINISHED]: vertexName=CE701424E12E45B29BC188054EB160E3, taskId=task_1452786540762_0001_1_01_000000, startTime=1452786541024, finishTime=1452786541065, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1452786540762_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541048, Process_Duration=7, Process_End_Time=1452786541055, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:01,065 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786540762_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:01,066 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3] : 1
2016-01-14 15:49:01,066 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:01,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:01,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:VERTEX_FINISHED]: vertexName=CE701424E12E45B29BC188054EB160E3, vertexId=vertex_1452786540762_0001_1_01, initRequestedTime=1452786540922, initedTime=1452786540931, startRequestedTime=1452786540937, startedTime=1452786540937, finishTime=1452786541066, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=496500736, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541048, Process_Duration=7, Process_End_Time=1452786541055, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786541024, firstTasksToStart=[ task_1452786540762_0001_1_01_000000 ], lastTaskFinishTime=1452786541063, lastTasksToFinish=[ task_1452786540762_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786540762_0001_1_01_000000 ], longestDurationTasks=[ task_1452786540762_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:01,067 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:01,067 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:01,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786540762_0001_00_000002, stoppedTime=1452786541067, exitStatus=0
2016-01-14 15:49:01,067 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:01,068 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:01,068 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786540762_0001_1
2016-01-14 15:49:01,068 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:01,068 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786540762_0001_1
2016-01-14 15:49:01,068 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CD5592991BCD48A983552E23A4C6D9FC for vertex: vertex_1452786540762_0001_1_01 [CE701424E12E45B29BC188054EB160E3]
2016-01-14 15:49:01,074 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786540762_0001_1_00 [FA377A46CEE84747BD0F6CBB8377AAE6]
2016-01-14 15:49:01,074 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786540762_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:49:01,075 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786540762_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:01,075 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786540762_0001_00_000002, Exiting
2016-01-14 15:49:01,075 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786540762_0001_00_000002
2016-01-14 15:49:01,075 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786540762_0001_00_000002
2016-01-14 15:49:01,075 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786540762_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:01,075 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786540762_0001_00_000002 completed successfully
2016-01-14 15:49:01,077 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786540762_0001_1, queueSize=0
2016-01-14 15:49:01,077 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:49:01,077 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786540762_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786540762_0001_1, startTime=1452786540919, finishTime=1452786541074, timeTaken=155, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=993001472, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573082046, Process_Duration=12, Process_End_Time=2905573082058, Read_Duration=2, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:49:01,078 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786540762_0001_1 finished with state: SUCCEEDED
2016-01-14 15:49:01,078 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786540762_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:49:01,078 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786540762_0001_00_000002
2016-01-14 15:49:01,078 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786540762_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:49:01 Completed Dag: dag_1452786540762_0001_1
2016-01-14 15:49:01,078 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786540762_0001_1, dagState=SUCCEEDED
2016-01-14 15:49:01,079 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786540762_0001_1_post
2016-01-14 15:49:01,079 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,079 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:49:01,083 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_false_true_true, applicationId=application_1452786540762_0001
2016-01-14 15:49:01,084 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:49:01,085 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:49:01,085 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:49:01,085 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:49:01,085 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:49:01,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:49:01,085 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:49:01,086 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:49:01,086 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:49:01,086 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:49:01,086 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:49:01,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:49:01,087 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:49:01,091 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/16AD979320D94C9494E2E800AB83F2BF/AB9DB22E328E4BB7863DF7B873241804/.tez/application_1452786540762_0001
2016-01-14 15:49:01,092 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.344
2016-01-14 15:49:01,141 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:49:01,142 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:49:01,171 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 15:49:01,178 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 15:49:01,179 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:49:01,179 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:49:01,179 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:49:01,196 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:49:01,201 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true
2016-01-14 15:49:01,235 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:49:01,235 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:01,236 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true"]
2016-01-14 15:49:01,236 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:49:01,237 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:49:01,237 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:49:01,238 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...nsive/true_true_true_true
2016-01-14 15:49:01,238 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:49:01,246 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:49:01,251 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:49:01,251 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:49:01,251 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:49:01,259 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387/.tez/application_1452786541251_0001 doesn't exist and is created
2016-01-14 15:49:01,274 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:49:01,274 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:49:01,275 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387/.tez/application_1452786541251_0001
2016-01-14 15:49:01,277 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786541251_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:49:01,286 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:49:01,286 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:49:01,288 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:49:01,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:49:01,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387/.tez/application_1452786541251_0001/localmode-log-dir
2016-01-14 15:49:01,290 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387/.tez/application_1452786541251_0001/localmode-log-dir/history.txt.appattempt_1452786541251_0001_000000, maxErrors=10
2016-01-14 15:49:01,290 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:49:01,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786541251_0001_000000, appSubmitTime=1452786541277, launchTime=1452786541277
2016-01-14 15:49:01,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:49:01,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:49:01,296 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:49:01,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:49:01,300 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:50977
2016-01-14 15:49:01,301 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:49:01,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786541251_0001_000000, startTime=1452786541303
2016-01-14 15:49:01,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:49:01,303 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:49:01,375 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:49:01,375 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:49:01,375 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...nsive/true_true_true_true, applicationId=application_1452786541251_0001, dagName=[B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387] comprehensivesort/(1/1) ...nsive/true_true_true_true
2016-01-14 15:49:01,387 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387] comprehensivesort/(1/1) ...nsive/true_true_true_true
2016-01-14 15:49:01,388 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786541251_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387/.tez/application_1452786541251_0001/localmode-log-dir/dag_1452786541251_0001_1.dot
2016-01-14 15:49:01,388 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387/.tez/application_1452786541251_0001/localmode-log-dir/dag_1452786541251_0001_1-tez-dag.pb.txt
2016-01-14 15:49:01,395 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786541251_0001_1
2016-01-14 15:49:01,395 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,396 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387] comprehensivesort/(1/1) ...nsive/true_true_true_true
2016-01-14 15:49:01 Running Dag: dag_1452786541251_0001_1
2016-01-14 15:49:01,407 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:49:01,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786541251_0001_1, submitTime=1452786541387
2016-01-14 15:49:01,408 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AB4C7DF5DA8447749FE8345AFB11FBC8
2016-01-14 15:49:01,408 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:49:01,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:49:01,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786541251_0001_1, initTime=1452786541407
2016-01-14 15:49:01,409 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786541251_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:49:01,409 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:DAG_STARTED]: dagID=dag_1452786541251_0001_1, startTime=1452786541409
2016-01-14 15:49:01,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:49:01,409 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...nsive/true_true_true_true, applicationId=application_1452786541251_0001, dagName=[B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387] comprehensivesort/(1/1) ...nsive/true_true_true_true
2016-01-14 15:49:01,409 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:49:01,409 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786541251_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:49:01,410 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C541266DE6344EAC8A754F044C5E8CDF : {D7D1AAAE281F4130BF53B0F54E01892D={InputName=D7D1AAAE281F4130BF53B0F54E01892D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:49:01,410 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D7D1AAAE281F4130BF53B0F54E01892D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:49:01,410 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,410 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,410 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,410 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]. Starting root input initializers: 1
2016-01-14 15:49:01,411 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:49:01,411 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,412 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D7D1AAAE281F4130BF53B0F54E01892D on vertex vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,414 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:49:01,414 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D7D1AAAE281F4130BF53B0F54E01892D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:49:01,414 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:49:01,415 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,415 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:49:01,415 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:49:01,415 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,416 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:49:01,417 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D7D1AAAE281F4130BF53B0F54E01892D on vertex vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,417 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,418 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0F272849C0D9471B9F27E73D2A02391C, vertexId=vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:49:01,418 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0F272849C0D9471B9F27E73D2A02391C, vertexId=vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,420 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0F272849C0D9471B9F27E73D2A02391C, vertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:49:01,420 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0F272849C0D9471B9F27E73D2A02391C, vertexId=vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,421 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786541251_0001, with dag id: dag_1452786541251_0001_1
2016-01-14 15:49:01,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, vertexId=vertex_1452786541251_0001_1_01, initRequestedTime=1452786541411, initedTime=1452786541422, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:49:01,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:49:01,422 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:49:01,423 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:49:01,423 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D7D1AAAE281F4130BF53B0F54E01892D=forAllWorkUnits=true, update=[1]}
2016-01-14 15:49:01,423 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] parallelism set to 1
2016-01-14 15:49:01,424 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:49:01,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786541251_0001_1_00, eventCount=1
2016-01-14 15:49:01,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:49:01,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C541266DE6344EAC8A754F044C5E8CDF, vertexId=vertex_1452786541251_0001_1_00, initRequestedTime=1452786541409, initedTime=1452786541425, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:49:01,425 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] with distanceFromRoot: 0
2016-01-14 15:49:01,426 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:49:01,426 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,427 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:49:01,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786541251_0001_1_00, startRequestedTime=1452786541426, startedTime=1452786541426
2016-01-14 15:49:01,428 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:49:01,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:01,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:TASK_STARTED]: vertexName=C541266DE6344EAC8A754F044C5E8CDF, taskId=task_1452786541251_0001_1_00_000000, scheduledTime=1452786541428, launchTime=1452786541428
2016-01-14 15:49:01,429 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541251_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:01,430 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786541251_0001_1_00 for vertex: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8] numStartedSources: 1 numSources: 1
2016-01-14 15:49:01,430 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8] with distanceFromRoot: 1
2016-01-14 15:49:01,431 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,431 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C541266DE6344EAC8A754F044C5E8CDF in vertex: AB4C7DF5DA8447749FE8345AFB11FBC8
2016-01-14 15:49:01,431 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AB4C7DF5DA8447749FE8345AFB11FBC8 with 1 source tasks and 1 pending tasks
2016-01-14 15:49:01,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:49:01,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786541251_0001_1_01, startRequestedTime=1452786541431, startedTime=1452786541431
2016-01-14 15:49:01,432 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:49:01,433 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387] comprehensivesort/(1/1) ...nsive/true_true_true_true, VertexName: C541266DE6344EAC8A754F044C5E8CDF, VertexParallelism: 1, TaskAttemptID:attempt_1452786541251_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7D1AAAE281F4130BF53B0F54E01892D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:49:01,434 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541251_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:01,434 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:01,435 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:49:01,435 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:49:01,435 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:49:01,435 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786541251_0001_00_000001
2016-01-14 15:49:01,435 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:01,437 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786541251_0001_00_000001
2016-01-14 15:49:01,437 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:49:01,437 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:01,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786541251_0001_00_000001, launchTime=1452786541437
2016-01-14 15:49:01,437 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:01,438 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786541251_0001_00_000001
2016-01-14 15:49:01,438 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786541251_0001_1_00_000000_0] to container: [container_1452786541251_0001_00_000001]
2016-01-14 15:49:01,438 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:01,438 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786541251_0001_00_000001 given task: attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,439 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786541251_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,439 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786541251_0001_1_00_000000_0] started. Is using containerId: [container_1452786541251_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:49:01 Starting to run new task attempt: attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,439 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,439 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,439 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:01,440 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:01,440 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:01,440 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:01,440 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:01,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387] comprehensivesort/(1/1) ...nsive/true_true_true_true, VertexName: C541266DE6344EAC8A754F044C5E8CDF, VertexParallelism: 1, TaskAttemptID:attempt_1452786541251_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7D1AAAE281F4130BF53B0F54E01892D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:49:01,440 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:01,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C541266DE6344EAC8A754F044C5E8CDF, taskAttemptId=attempt_1452786541251_0001_1_00_000000_0, startTime=1452786541439, containerId=container_1452786541251_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786541251_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:49:01,441 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541251_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:01,440 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:01,441 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541251_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:01,442 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:01,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,442 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786541251_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:49:01,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D7D1AAAE281F4130BF53B0F54E01892D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:49:01,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:49:01,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:01,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D7D1AAAE281F4130BF53B0F54E01892D
2016-01-14 15:49:01,443 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D7D1AAAE281F4130BF53B0F54E01892D
2016-01-14 15:49:01,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:01,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AB4C7DF5DA8447749FE8345AFB11FBC8
2016-01-14 15:49:01,445 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:49:01,445 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:01,445 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D7D1AAAE281F4130BF53B0F54E01892D
2016-01-14 15:49:01,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D7D1AAAE281F4130BF53B0F54E01892D
2016-01-14 15:49:01,446 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:49:01,446 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AB4C7DF5DA8447749FE8345AFB11FBC8
2016-01-14 15:49:01,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AB4C7DF5DA8447749FE8345AFB11FBC8
2016-01-14 15:49:01,452 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C541266DE6344EAC8A754F044C5E8CDF, ordinal: 0
2016-01-14 15:49:01,452 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C541266DE6344EAC8A754F044C5E8CDF, mem on start (mb), free: 123, total: 473, max: 672
2016-01-14 15:49:01,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:01,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:01,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:01,453 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:01,453 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:49:01,454 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:01,454 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:49:01,454 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:01,454 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:49:01,454 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D7D1AAAE281F4130BF53B0F54E01892D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:49:01,454 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AB4C7DF5DA8447749FE8345AFB11FBC8, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:49:01,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:01,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D7D1AAAE281F4130BF53B0F54E01892D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:01,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D7D1AAAE281F4130BF53B0F54E01892D
2016-01-14 15:49:01,455 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:49:01,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:01,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:01,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D7D1AAAE281F4130BF53B0F54E01892D
2016-01-14 15:49:01,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:01,456 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,456 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:49:01,456 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@a4146e8
2016-01-14 15:49:01,457 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@a4146e8
2016-01-14 15:49:01,457 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:49:01,457 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:49:01,459 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:01,459 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: D7D1AAAE281F4130BF53B0F54E01892D
2016-01-14 15:49:01,460 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: A057EE1D116541918D0408B1EA77E1CE
2016-01-14 15:49:01,460 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A057EE1D116541918D0408B1EA77E1CE
2016-01-14 15:49:01,462 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@70497e11; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@68f3c7ae; comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@5cbaa656; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:49:01,462 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:49:01,462 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:49:01,462 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:01,472 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:49:01,472 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:49:01,485 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:49:01,485 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:49:01,485 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:49:01,485 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:49:01,485 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:49:01,486 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:01,486 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:49:01,486 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C541266DE6344EAC8A754F044C5E8CDF, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:49:01,490 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,491 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C541266DE6344EAC8A754F044C5E8CDF, mem on close (mb), free: 212, total: 474, max: 672
2016-01-14 15:49:01,491 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:49:01,491 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:49:01,491 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:49:01,491 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:49:01,494 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:49:01 Completed running task attempt: attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,497 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786541251_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:01,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786541251_0001_1_00, eventCount=1
2016-01-14 15:49:01,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C541266DE6344EAC8A754F044C5E8CDF, taskAttemptId=attempt_1452786541251_0001_1_00_000000_0, startTime=1452786541439, finishTime=1452786541498, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541487, Process_Duration=3, Process_End_Time=1452786541490, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:01,498 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541251_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:01,498 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786541251_0001_1_00_000000_0
2016-01-14 15:49:01,498 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:01,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:01,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:TASK_FINISHED]: vertexName=C541266DE6344EAC8A754F044C5E8CDF, taskId=task_1452786541251_0001_1_00_000000, startTime=1452786541439, finishTime=1452786541498, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1452786541251_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541487, Process_Duration=3, Process_End_Time=1452786541490, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:01,499 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541251_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:01,499 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] : 1
2016-01-14 15:49:01,499 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:01,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:01,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:VERTEX_FINISHED]: vertexName=C541266DE6344EAC8A754F044C5E8CDF, vertexId=vertex_1452786541251_0001_1_00, initRequestedTime=1452786541409, initedTime=1452786541425, startRequestedTime=1452786541426, startedTime=1452786541426, finishTime=1452786541499, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541487, Process_Duration=3, Process_End_Time=1452786541490, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786541439, firstTasksToStart=[ task_1452786541251_0001_1_00_000000 ], lastTaskFinishTime=1452786541498, lastTasksToFinish=[ task_1452786541251_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786541251_0001_1_00_000000 ], longestDurationTasks=[ task_1452786541251_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:01,500 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:01,501 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:01,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786541251_0001_00_000001, stoppedTime=1452786541501, exitStatus=0
2016-01-14 15:49:01,501 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:01,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:01,501 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8] attempt: attempt_1452786541251_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:49:01,501 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AB4C7DF5DA8447749FE8345AFB11FBC8
2016-01-14 15:49:01,502 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,502 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:01,502 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786541251_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:49:01,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:TASK_STARTED]: vertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, taskId=task_1452786541251_0001_1_01_000000, scheduledTime=1452786541502, launchTime=1452786541502
2016-01-14 15:49:01,502 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541251_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:01,502 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,503 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:49:01,503 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387] comprehensivesort/(1/1) ...nsive/true_true_true_true, VertexName: AB4C7DF5DA8447749FE8345AFB11FBC8, VertexParallelism: 1, TaskAttemptID:attempt_1452786541251_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C541266DE6344EAC8A754F044C5E8CDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0F272849C0D9471B9F27E73D2A02391C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:01,503 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541251_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:01,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1284, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786541487, Process_Duration=3, Process_End_Time=1452786541490, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:49:01,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:01,503 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:01,503 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:01,503 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,504 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786541251_0001_00_000001
2016-01-14 15:49:01,504 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786541251_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:49:01,504 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786541251_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:01,504 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786541251_0001_00_000001, Exiting
2016-01-14 15:49:01,504 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786541251_0001_00_000001
2016-01-14 15:49:01,504 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786541251_0001_00_000001
2016-01-14 15:49:01,505 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786541251_0001_00_000002
2016-01-14 15:49:01,505 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:01,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786541251_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:01,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786541251_0001_00_000001 completed successfully
2016-01-14 15:49:01,506 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:49:01,507 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786541251_0001_00_000002
2016-01-14 15:49:01,507 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:49:01,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786541251_0001_00_000002, launchTime=1452786541510
2016-01-14 15:49:01,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:01,510 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:01,511 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786541251_0001_00_000002
2016-01-14 15:49:01,511 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786541251_0001_1_01_000000_0] to container: [container_1452786541251_0001_00_000002]
2016-01-14 15:49:01,512 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:01,512 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786541251_0001_1_01_000000_0] started. Is using containerId: [container_1452786541251_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:49:01,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, taskAttemptId=attempt_1452786541251_0001_1_01_000000_0, startTime=1452786541512, containerId=container_1452786541251_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786541251_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:49:01,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:01,513 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541251_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:01,513 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541251_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:01,513 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786541251_0001_00_000002 given task: attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,514 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786541251_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01 Starting to run new task attempt: attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,514 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,514 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,514 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:01,515 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:01,515 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:01,515 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:01,515 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:01,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387] comprehensivesort/(1/1) ...nsive/true_true_true_true, VertexName: AB4C7DF5DA8447749FE8345AFB11FBC8, VertexParallelism: 1, TaskAttemptID:attempt_1452786541251_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C541266DE6344EAC8A754F044C5E8CDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0F272849C0D9471B9F27E73D2A02391C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:01,515 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:01,519 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:01,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,520 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786541251_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:49:01,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C541266DE6344EAC8A754F044C5E8CDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:49:01,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:01,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,524 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:49:01,524 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:49:01,524 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,529 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:01,535 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AB4C7DF5DA8447749FE8345AFB11FBC8, ordinal: 1
2016-01-14 15:49:01,536 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AB4C7DF5DA8447749FE8345AFB11FBC8, mem on start (mb), free: 207, total: 474, max: 672
2016-01-14 15:49:01,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:01,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:01,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0F272849C0D9471B9F27E73D2A02391C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:49:01,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:01,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0F272849C0D9471B9F27E73D2A02391C
2016-01-14 15:49:01,538 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:49:01,538 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0F272849C0D9471B9F27E73D2A02391C
2016-01-14 15:49:01,547 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:49:01,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0F272849C0D9471B9F27E73D2A02391C
2016-01-14 15:49:01,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:01,548 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:01,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:49:01,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:49:01,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:01,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:01,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:49:01,549 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C541266DE6344EAC8A754F044C5E8CDF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:49:01,549 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0F272849C0D9471B9F27E73D2A02391C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:49:01,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:01,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C541266DE6344EAC8A754F044C5E8CDF being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:01,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:01,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:01,550 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:01,550 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:49:01,550 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C541266DE6344EAC8A754F044C5E8CDF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:49:01,551 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:49:01,551 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:49:01,551 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:49:01,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:01,556 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,557 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786541251_0001_1_00_000000_0_10155, runDuration: 52182, ]
2016-01-14 15:49:01,558 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786541251_0001_1_00_000000_0_10155] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786541558, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:49:01,559 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,559 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:49:01,559 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C541266DE6344EAC8A754F044C5E8CDF] #27 in 2ms
2016-01-14 15:49:01,559 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:49:01,560 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: A057EE1D116541918D0408B1EA77E1CE
2016-01-14 15:49:01,560 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true"], id: 0F272849C0D9471B9F27E73D2A02391C
2016-01-14 15:49:01,560 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C541266DE6344EAC8A754F044C5E8CDF] #27, status:false, isInterrupted:false
2016-01-14 15:49:01,560 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true"]
2016-01-14 15:49:01,561 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A057EE1D116541918D0408B1EA77E1CE, for 1 inputs
2016-01-14 15:49:01,561 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:49:01,562 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:49:01,562 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:01,562 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:49:01,563 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:49:01,563 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,563 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AB4C7DF5DA8447749FE8345AFB11FBC8, all 1 inputs ready in: 00:00:00.002
2016-01-14 15:49:01,563 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:49:01,565 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@7efe22aa; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5e960808
2016-01-14 15:49:01,569 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:49:01,569 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:49:01,569 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,570 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786541251_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:49:01,570 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,571 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AB4C7DF5DA8447749FE8345AFB11FBC8, mem on close (mb), free: 204, total: 474, max: 672
2016-01-14 15:49:01,571 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C541266DE6344EAC8A754F044C5E8CDF
2016-01-14 15:49:01,571 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:49:01,572 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:49:01,572 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:01,573 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:49:01,573 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:49:01 Completed running task attempt: attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,574 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786541251_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:01,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786541563, Process_Duration=7, Process_End_Time=1452786541570, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:49:01,575 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:01,575 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:01,575 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, taskAttemptId=attempt_1452786541251_0001_1_01_000000_0, startTime=1452786541512, finishTime=1452786541574, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541563, Process_Duration=7, Process_End_Time=1452786541570, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:01,575 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541251_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:01,576 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:01,576 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786541251_0001_1_01_000000_0
2016-01-14 15:49:01,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:TASK_FINISHED]: vertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, taskId=task_1452786541251_0001_1_01_000000, startTime=1452786541512, finishTime=1452786541576, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1452786541251_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541563, Process_Duration=7, Process_End_Time=1452786541570, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:01,576 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786541251_0001_00_000002
2016-01-14 15:49:01,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:01,576 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541251_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:01,577 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8] : 1
2016-01-14 15:49:01,577 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:VERTEX_FINISHED]: vertexName=AB4C7DF5DA8447749FE8345AFB11FBC8, vertexId=vertex_1452786541251_0001_1_01, initRequestedTime=1452786541411, initedTime=1452786541422, startRequestedTime=1452786541431, startedTime=1452786541431, finishTime=1452786541577, timeTaken=146, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541563, Process_Duration=7, Process_End_Time=1452786541570, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786541512, firstTasksToStart=[ task_1452786541251_0001_1_01_000000 ], lastTaskFinishTime=1452786541574, lastTasksToFinish=[ task_1452786541251_0001_1_01_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786541251_0001_1_01_000000 ], longestDurationTasks=[ task_1452786541251_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:01,578 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:01,578 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:01,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786541251_0001_00_000002, stoppedTime=1452786541578, exitStatus=0
2016-01-14 15:49:01,578 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:01,578 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:01,578 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786541251_0001_1
2016-01-14 15:49:01,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786541251_0001_1
2016-01-14 15:49:01,579 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786541251_0001_1_00 [C541266DE6344EAC8A754F044C5E8CDF]
2016-01-14 15:49:01,579 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0F272849C0D9471B9F27E73D2A02391C for vertex: vertex_1452786541251_0001_1_01 [AB4C7DF5DA8447749FE8345AFB11FBC8]
2016-01-14 15:49:01,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:01,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:01,587 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786541251_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:49:01,587 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786541251_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:01,587 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786541251_0001_00_000002, Exiting
2016-01-14 15:49:01,587 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786541251_0001_00_000002
2016-01-14 15:49:01,588 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786541251_0001_00_000002
2016-01-14 15:49:01,588 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786541251_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:01,588 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786541251_0001_1, queueSize=0
2016-01-14 15:49:01,588 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786541251_0001_00_000002 completed successfully
2016-01-14 15:49:01,588 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:49:01,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541251_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786541251_0001_1, startTime=1452786541409, finishTime=1452786541585, timeTaken=176, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=1518, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=994050048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573083050, Process_Duration=10, Process_End_Time=2905573083060, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:49:01,589 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786541251_0001_1 finished with state: SUCCEEDED
2016-01-14 15:49:01,589 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786541251_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:49:01,589 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786541251_0001_00_000002
2016-01-14 15:49:01,589 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541251_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:49:01 Completed Dag: dag_1452786541251_0001_1
2016-01-14 15:49:01,590 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786541251_0001_1, dagState=SUCCEEDED
2016-01-14 15:49:01,590 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786541251_0001_1_post
2016-01-14 15:49:01,591 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,591 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:49:01,593 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...nsive/true_true_true_true, applicationId=application_1452786541251_0001
2016-01-14 15:49:01,594 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:49:01,594 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:49:01,594 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:49:01,594 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:49:01,595 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:49:01,595 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:49:01,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:49:01,596 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:49:01,596 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:49:01,596 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:49:01,596 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:49:01,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:49:01,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:49:01,602 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B698BCB0371149938768A63501926A83/FA7C5D9C919C4E4BAAE8D8E9AA124387/.tez/application_1452786541251_0001
2016-01-14 15:49:01,602 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.367
2016-01-14 15:49:01,652 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:49:01,653 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:49:01,678 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 15:49:01,685 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 15:49:01,685 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:49:01,685 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:49:01,686 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:49:01,697 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:49:01,702 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true
2016-01-14 15:49:01,735 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:49:01,736 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:01,736 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true"]
2016-01-14 15:49:01,736 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] parallel execution of steps is enabled: false
2016-01-14 15:49:01,736 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executing total steps: 1
2016-01-14 15:49:01,736 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] allocating management threads: 1
2016-01-14 15:49:01,737 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...sive/false_true_true_true
2016-01-14 15:49:01,737 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:49:01,745 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:49:01,749 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:49:01,750 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:49:01,750 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 15:49:01,758 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F/.tez/application_1452786541750_0001 doesn't exist and is created
2016-01-14 15:49:01,775 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:49:01,775 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:49:01,776 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F/.tez/application_1452786541750_0001
2016-01-14 15:49:01,778 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786541750_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:49:01,786 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:49:01,787 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:49:01,788 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:49:01,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:49:01,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F/.tez/application_1452786541750_0001/localmode-log-dir
2016-01-14 15:49:01,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F/.tez/application_1452786541750_0001/localmode-log-dir/history.txt.appattempt_1452786541750_0001_000000, maxErrors=10
2016-01-14 15:49:01,788 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:49:01,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786541750_0001_000000, appSubmitTime=1452786541778, launchTime=1452786541778
2016-01-14 15:49:01,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:49:01,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:49:01,793 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:49:01,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:49:01,794 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:47997
2016-01-14 15:49:01,796 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:49:01,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786541750_0001_000000, startTime=1452786541798
2016-01-14 15:49:01,798 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:49:01,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:49:01,875 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:49:01,876 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:49:01,876 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/false_true_true_true, applicationId=application_1452786541750_0001, dagName=[D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F] comprehensivesort/(1/1) ...sive/false_true_true_true
2016-01-14 15:49:01,888 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F] comprehensivesort/(1/1) ...sive/false_true_true_true
2016-01-14 15:49:01,889 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786541750_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F/.tez/application_1452786541750_0001/localmode-log-dir/dag_1452786541750_0001_1.dot
2016-01-14 15:49:01,890 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F/.tez/application_1452786541750_0001/localmode-log-dir/dag_1452786541750_0001_1-tez-dag.pb.txt
2016-01-14 15:49:01,896 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786541750_0001_1
2016-01-14 15:49:01,896 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,896 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F] comprehensivesort/(1/1) ...sive/false_true_true_true
2016-01-14 15:49:01 Running Dag: dag_1452786541750_0001_1
2016-01-14 15:49:01,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786541750_0001_1, submitTime=1452786541888
2016-01-14 15:49:01,906 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:49:01,906 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B53080FC39E14434B0AB1F95E46A64FF
2016-01-14 15:49:01,907 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:49:01,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786541750_0001_1, initTime=1452786541906
2016-01-14 15:49:01,907 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786541750_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:49:01,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:49:01,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:DAG_STARTED]: dagID=dag_1452786541750_0001_1, startTime=1452786541907
2016-01-14 15:49:01,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:49:01,907 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/false_true_true_true, applicationId=application_1452786541750_0001, dagName=[D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F] comprehensivesort/(1/1) ...sive/false_true_true_true
2016-01-14 15:49:01,907 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:49:01,908 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786541750_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:49:01,908 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D049DEF22B824A05934E514D37BD5E65 : {9B7C10685DD24D35B86E2C601467D61C={InputName=9B7C10685DD24D35B86E2C601467D61C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:49:01,908 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9B7C10685DD24D35B86E2C601467D61C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:49:01,908 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:01,908 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:01,908 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:01,908 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]. Starting root input initializers: 1
2016-01-14 15:49:01,909 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:49:01,909 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:01,909 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9B7C10685DD24D35B86E2C601467D61C on vertex vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:01,911 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:49:01,911 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9B7C10685DD24D35B86E2C601467D61C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:49:01,911 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:49:01,912 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:49:01,912 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:01,912 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:01,912 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:01,912 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=290C215B44E24E73A7C6D899AE227D28, vertexId=vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:49:01,912 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=290C215B44E24E73A7C6D899AE227D28, vertexId=vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:01,912 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:49:01,914 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=290C215B44E24E73A7C6D899AE227D28, vertexName=B53080FC39E14434B0AB1F95E46A64FF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:49:01,914 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=290C215B44E24E73A7C6D899AE227D28, vertexId=vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:01,915 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:49:01,915 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9B7C10685DD24D35B86E2C601467D61C on vertex vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:01,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:49:01,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B53080FC39E14434B0AB1F95E46A64FF, vertexId=vertex_1452786541750_0001_1_01, initRequestedTime=1452786541909, initedTime=1452786541915, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:49:01,916 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:49:01,916 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:49:01,916 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9B7C10685DD24D35B86E2C601467D61C=forAllWorkUnits=true, update=[1]}
2016-01-14 15:49:01,916 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] parallelism set to 1
2016-01-14 15:49:01,916 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:49:01,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786541750_0001_1_00, eventCount=1
2016-01-14 15:49:01,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D049DEF22B824A05934E514D37BD5E65, vertexId=vertex_1452786541750_0001_1_00, initRequestedTime=1452786541908, initedTime=1452786541917, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:49:01,917 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] with distanceFromRoot: 0
2016-01-14 15:49:01,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:49:01,917 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:49:01,918 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:01,918 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:01,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:49:01,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786541750_0001_1_00, startRequestedTime=1452786541917, startedTime=1452786541917
2016-01-14 15:49:01,918 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:49:01,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:01,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:TASK_STARTED]: vertexName=D049DEF22B824A05934E514D37BD5E65, taskId=task_1452786541750_0001_1_00_000000, scheduledTime=1452786541918, launchTime=1452786541918
2016-01-14 15:49:01,919 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541750_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:01,919 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786541750_0001_1_00 for vertex: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF] numStartedSources: 1 numSources: 1
2016-01-14 15:49:01,919 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF] with distanceFromRoot: 1
2016-01-14 15:49:01,919 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:01,920 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D049DEF22B824A05934E514D37BD5E65 in vertex: B53080FC39E14434B0AB1F95E46A64FF
2016-01-14 15:49:01,920 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B53080FC39E14434B0AB1F95E46A64FF with 1 source tasks and 1 pending tasks
2016-01-14 15:49:01,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:49:01,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786541750_0001_1_01, startRequestedTime=1452786541919, startedTime=1452786541919
2016-01-14 15:49:01,920 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:49:01,920 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F] comprehensivesort/(1/1) ...sive/false_true_true_true, VertexName: D049DEF22B824A05934E514D37BD5E65, VertexParallelism: 1, TaskAttemptID:attempt_1452786541750_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9B7C10685DD24D35B86E2C601467D61C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B53080FC39E14434B0AB1F95E46A64FF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:49:01,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541750_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:01,921 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:01,921 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:49:01,922 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:49:01,922 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:49:01,922 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786541750_0001_00_000001
2016-01-14 15:49:01,922 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:01,923 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786541750_0001_00_000001
2016-01-14 15:49:01,923 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:49:01,924 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:01,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:01,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786541750_0001_00_000001, launchTime=1452786541924
2016-01-14 15:49:01,925 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786541750_0001_00_000001
2016-01-14 15:49:01,925 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786541750_0001_1_00_000000_0] to container: [container_1452786541750_0001_00_000001]
2016-01-14 15:49:01,925 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:01,926 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786541750_0001_1_00_000000_0] started. Is using containerId: [container_1452786541750_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:49:01,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D049DEF22B824A05934E514D37BD5E65, taskAttemptId=attempt_1452786541750_0001_1_00_000000_0, startTime=1452786541926, containerId=container_1452786541750_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786541750_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:49:01,926 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541750_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:01,926 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541750_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:01,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:01,927 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786541750_0001_00_000001 given task: attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:01,927 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786541750_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:01 Starting to run new task attempt: attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:01,928 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:01,928 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:01,928 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:01,928 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:01,928 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:01,928 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:01,928 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:01,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F] comprehensivesort/(1/1) ...sive/false_true_true_true, VertexName: D049DEF22B824A05934E514D37BD5E65, VertexParallelism: 1, TaskAttemptID:attempt_1452786541750_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9B7C10685DD24D35B86E2C601467D61C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B53080FC39E14434B0AB1F95E46A64FF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:49:01,928 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:01,930 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786541750_0001, with dag id: dag_1452786541750_0001_1
2016-01-14 15:49:01,931 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:01,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:01,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,932 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786541750_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:49:01,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9B7C10685DD24D35B86E2C601467D61C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:49:01,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:01,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9B7C10685DD24D35B86E2C601467D61C
2016-01-14 15:49:01,933 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9B7C10685DD24D35B86E2C601467D61C
2016-01-14 15:49:01,935 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:49:01,936 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9B7C10685DD24D35B86E2C601467D61C
2016-01-14 15:49:01,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9B7C10685DD24D35B86E2C601467D61C
2016-01-14 15:49:01,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,939 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:01,946 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D049DEF22B824A05934E514D37BD5E65, ordinal: 0
2016-01-14 15:49:01,946 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D049DEF22B824A05934E514D37BD5E65, mem on start (mb), free: 123, total: 474, max: 672
2016-01-14 15:49:01,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:01,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:01,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:01,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B53080FC39E14434B0AB1F95E46A64FF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:49:01,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:01,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B53080FC39E14434B0AB1F95E46A64FF
2016-01-14 15:49:01,950 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:49:01,950 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B53080FC39E14434B0AB1F95E46A64FF
2016-01-14 15:49:01,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B53080FC39E14434B0AB1F95E46A64FF
2016-01-14 15:49:01,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:01,951 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:01,952 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:49:01,952 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:01,953 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:49:01,953 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:01,953 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:49:01,954 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9B7C10685DD24D35B86E2C601467D61C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:49:01,954 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B53080FC39E14434B0AB1F95E46A64FF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:49:01,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:01,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9B7C10685DD24D35B86E2C601467D61C
2016-01-14 15:49:01,955 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:49:01,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9B7C10685DD24D35B86E2C601467D61C
2016-01-14 15:49:01,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9B7C10685DD24D35B86E2C601467D61C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:01,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:01,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:01,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:01,956 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:01,956 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:49:01,959 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:01,960 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5b2d569f
2016-01-14 15:49:01,960 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 9B7C10685DD24D35B86E2C601467D61C
2016-01-14 15:49:01,960 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 4832FE513AD6494AAC0D34DC5F819230
2016-01-14 15:49:01,960 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 4832FE513AD6494AAC0D34DC5F819230
2016-01-14 15:49:01,961 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5b2d569f
2016-01-14 15:49:01,961 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:49:01,961 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:49:01,961 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@3bc2c1d; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@36f425d5; comparator=cascading.tuple.hadoop.util.ReverseGroupingSortingComparator@56dfdf22; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:49:01,962 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:49:01,962 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:49:01,962 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:01,972 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:49:01,972 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:49:01,986 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:49:01,986 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:49:01,986 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:49:01,986 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:49:01,986 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:49:01,987 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:49:01,987 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:49:01,987 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D049DEF22B824A05934E514D37BD5E65, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:49:01,992 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:01,992 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D049DEF22B824A05934E514D37BD5E65, mem on close (mb), free: 211, total: 474, max: 672
2016-01-14 15:49:01,992 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:49:01,992 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:49:01,992 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:49:01,992 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:49:01,997 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:49:02 Completed running task attempt: attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:02,001 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786541750_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:02,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786541750_0001_1_00, eventCount=1
2016-01-14 15:49:02,002 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D049DEF22B824A05934E514D37BD5E65, taskAttemptId=attempt_1452786541750_0001_1_00_000000_0, startTime=1452786541926, finishTime=1452786542001, timeTaken=75, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541987, Process_Duration=5, Process_End_Time=1452786541992, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:02,002 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541750_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:02,002 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:02,002 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786541750_0001_1_00_000000_0
2016-01-14 15:49:02,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:TASK_FINISHED]: vertexName=D049DEF22B824A05934E514D37BD5E65, taskId=task_1452786541750_0001_1_00_000000, startTime=1452786541926, finishTime=1452786542002, timeTaken=76, status=SUCCEEDED, successfulAttemptID=attempt_1452786541750_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541987, Process_Duration=5, Process_End_Time=1452786541992, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:49:02,003 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541750_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:02,003 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] : 1
2016-01-14 15:49:02,003 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:02,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:VERTEX_FINISHED]: vertexName=D049DEF22B824A05934E514D37BD5E65, vertexId=vertex_1452786541750_0001_1_00, initRequestedTime=1452786541908, initedTime=1452786541917, startRequestedTime=1452786541917, startedTime=1452786541917, finishTime=1452786542003, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786541987, Process_Duration=5, Process_End_Time=1452786541992, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786541926, firstTasksToStart=[ task_1452786541750_0001_1_00_000000 ], lastTaskFinishTime=1452786542001, lastTasksToFinish=[ task_1452786541750_0001_1_00_000000 ], minTaskDuration=75, maxTaskDuration=75, avgTaskDuration=75.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786541750_0001_1_00_000000 ], longestDurationTasks=[ task_1452786541750_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:02,004 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:02,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:02,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:02,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:02,006 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:02,006 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF] attempt: attempt_1452786541750_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:49:02,006 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B53080FC39E14434B0AB1F95E46A64FF
2016-01-14 15:49:02,007 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:02,007 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:02,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:02,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786541750_0001_00_000001, stoppedTime=1452786542007, exitStatus=0
2016-01-14 15:49:02,008 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:02,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:TASK_STARTED]: vertexName=B53080FC39E14434B0AB1F95E46A64FF, taskId=task_1452786541750_0001_1_01_000000, scheduledTime=1452786542008, launchTime=1452786542008
2016-01-14 15:49:02,009 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541750_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:49:02,009 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786541750_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:49:02,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:49:02,009 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:02,010 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:49:02,010 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F] comprehensivesort/(1/1) ...sive/false_true_true_true, VertexName: B53080FC39E14434B0AB1F95E46A64FF, VertexParallelism: 1, TaskAttemptID:attempt_1452786541750_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D049DEF22B824A05934E514D37BD5E65, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=290C215B44E24E73A7C6D899AE227D28, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:02,010 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541750_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:49:02,010 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:49:02,011 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786541750_0001_00_000002
2016-01-14 15:49:02,011 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:49:02,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786541987, Process_Duration=5, Process_End_Time=1452786541992, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:49:02,012 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:02,012 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:02,012 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786541750_0001_00_000001
2016-01-14 15:49:02,012 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786541750_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:49:02,012 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786541750_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:02,012 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786541750_0001_00_000001, Exiting
2016-01-14 15:49:02,013 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786541750_0001_00_000001
2016-01-14 15:49:02,013 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786541750_0001_00_000001
2016-01-14 15:49:02,013 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452786541750_0001_00_000002
2016-01-14 15:49:02,014 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:49:02,014 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786541750_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:02,014 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786541750_0001_00_000001 completed successfully
2016-01-14 15:49:02,015 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:49:02,016 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:49:02,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:49:02,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786541750_0001_00_000002, launchTime=1452786542016
2016-01-14 15:49:02,017 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786541750_0001_00_000002
2016-01-14 15:49:02,017 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786541750_0001_1_01_000000_0] to container: [container_1452786541750_0001_00_000002]
2016-01-14 15:49:02,017 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:49:02,017 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786541750_0001_00_000002 given task: attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,018 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786541750_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,018 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786541750_0001_1_01_000000_0] started. Is using containerId: [container_1452786541750_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:49:02,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B53080FC39E14434B0AB1F95E46A64FF, taskAttemptId=attempt_1452786541750_0001_1_01_000000_0, startTime=1452786542018, containerId=container_1452786541750_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786541750_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:49:02 Starting to run new task attempt: attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,018 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541750_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:49:02,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:49:02,018 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541750_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:49:02,018 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,019 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:02,019 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:49:02,019 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:49:02,019 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:49:02,019 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:49:02,019 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:49:02,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F] comprehensivesort/(1/1) ...sive/false_true_true_true, VertexName: B53080FC39E14434B0AB1F95E46A64FF, VertexParallelism: 1, TaskAttemptID:attempt_1452786541750_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D049DEF22B824A05934E514D37BD5E65, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=290C215B44E24E73A7C6D899AE227D28, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:49:02,019 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:49:02,020 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:49:02,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:02,021 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786541750_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:49:02,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:02,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=290C215B44E24E73A7C6D899AE227D28, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:49:02,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:49:02,022 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 290C215B44E24E73A7C6D899AE227D28
2016-01-14 15:49:02,022 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:49:02,022 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 290C215B44E24E73A7C6D899AE227D28
2016-01-14 15:49:02,024 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:49:02,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D049DEF22B824A05934E514D37BD5E65, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:49:02,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:49:02,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:02,028 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:49:02,028 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:49:02,028 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:02,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:02,030 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B53080FC39E14434B0AB1F95E46A64FF, ordinal: 1
2016-01-14 15:49:02,030 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B53080FC39E14434B0AB1F95E46A64FF, mem on start (mb), free: 205, total: 474, max: 672
2016-01-14 15:49:02,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:49:02,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:49:02,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:49:02,034 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:49:02,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 290C215B44E24E73A7C6D899AE227D28
2016-01-14 15:49:02,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:49:02,034 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:49:02,034 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:49:02,034 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:49:02,034 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:49:02,034 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:49:02,035 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:49:02,035 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 290C215B44E24E73A7C6D899AE227D28, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:49:02,035 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D049DEF22B824A05934E514D37BD5E65, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:49:02,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:49:02,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D049DEF22B824A05934E514D37BD5E65 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:49:02,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:49:02,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:49:02,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:02,036 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:49:02,036 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:49:02,036 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D049DEF22B824A05934E514D37BD5E65 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:49:02,036 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:49:02,037 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:49:02,037 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:49:02,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:02,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:49:02,038 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,041 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:49:02,041 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786541750_0001_1_00_000000_0_10161, runDuration: 51069, ]
2016-01-14 15:49:02,041 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 4832FE513AD6494AAC0D34DC5F819230
2016-01-14 15:49:02,041 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true"], id: 290C215B44E24E73A7C6D899AE227D28
2016-01-14 15:49:02,041 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true"]
2016-01-14 15:49:02,042 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 4832FE513AD6494AAC0D34DC5F819230, for 1 inputs
2016-01-14 15:49:02,042 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786541750_0001_1_00_000000_0_10161] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786542042, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:49:02,043 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:02,043 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:49:02,043 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D049DEF22B824A05934E514D37BD5E65] #28 in 2ms
2016-01-14 15:49:02,043 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D049DEF22B824A05934E514D37BD5E65] #28, status:false, isInterrupted:false
2016-01-14 15:49:02,044 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:49:02,044 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:49:02,044 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:02,044 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:49:02,045 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:49:02,045 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:02,045 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:49:02,045 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B53080FC39E14434B0AB1F95E46A64FF, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:49:02,046 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@2b03bf8; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@311829
2016-01-14 15:49:02,051 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:49:02,051 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:49:02,052 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,052 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786541750_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:49:02,053 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,053 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B53080FC39E14434B0AB1F95E46A64FF, mem on close (mb), free: 203, total: 474, max: 672
2016-01-14 15:49:02,053 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D049DEF22B824A05934E514D37BD5E65
2016-01-14 15:49:02,054 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:49:02,054 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:49:02,054 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:49:02,055 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:49:02,055 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:49:02 Completed running task attempt: attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,056 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786541750_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:49:02,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:49:02,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B53080FC39E14434B0AB1F95E46A64FF, taskAttemptId=attempt_1452786541750_0001_1_01_000000_0, startTime=1452786542018, finishTime=1452786542056, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786542045, Process_Duration=8, Process_End_Time=1452786542053, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:02,057 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786541750_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:49:02,057 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:49:02,057 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786541750_0001_1_01_000000_0
2016-01-14 15:49:02,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:49:02,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:TASK_FINISHED]: vertexName=B53080FC39E14434B0AB1F95E46A64FF, taskId=task_1452786541750_0001_1_01_000000, startTime=1452786542018, finishTime=1452786542057, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1452786541750_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786542045, Process_Duration=8, Process_End_Time=1452786542053, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:49:02,058 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786541750_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:49:02,058 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF] : 1
2016-01-14 15:49:02,058 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:02,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786542045, Process_Duration=8, Process_End_Time=1452786542053, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:49:02,059 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 15:49:02,059 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:02,059 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:49:02,059 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786541750_0001_00_000002
2016-01-14 15:49:02,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:VERTEX_FINISHED]: vertexName=B53080FC39E14434B0AB1F95E46A64FF, vertexId=vertex_1452786541750_0001_1_01, initRequestedTime=1452786541909, initedTime=1452786541915, startRequestedTime=1452786541919, startedTime=1452786541919, finishTime=1452786542058, timeTaken=139, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=497025024, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786542045, Process_Duration=8, Process_End_Time=1452786542053, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786542018, firstTasksToStart=[ task_1452786541750_0001_1_01_000000 ], lastTaskFinishTime=1452786542056, lastTasksToFinish=[ task_1452786541750_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786541750_0001_1_01_000000 ], longestDurationTasks=[ task_1452786541750_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:49:02,059 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:49:02,060 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:49:02,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786541750_0001_00_000002, stoppedTime=1452786542060, exitStatus=0
2016-01-14 15:49:02,060 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:49:02,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:49:02,060 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:49:02,060 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786541750_0001_1
2016-01-14 15:49:02,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786541750_0001_1
2016-01-14 15:49:02,060 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 290C215B44E24E73A7C6D899AE227D28 for vertex: vertex_1452786541750_0001_1_01 [B53080FC39E14434B0AB1F95E46A64FF]
2016-01-14 15:49:02,068 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786541750_0001_1_00 [D049DEF22B824A05934E514D37BD5E65]
2016-01-14 15:49:02,069 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786541750_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:49:02,069 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786541750_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:49:02,070 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786541750_0001_00_000002, Exiting
2016-01-14 15:49:02,070 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786541750_0001_00_000002
2016-01-14 15:49:02,070 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786541750_0001_00_000002
2016-01-14 15:49:02,070 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786541750_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:49:02,070 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786541750_0001_00_000002 completed successfully
2016-01-14 15:49:02,070 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786541750_0001_1, queueSize=0
2016-01-14 15:49:02,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:49:02,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786541750_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786541750_0001_1, startTime=1452786541907, finishTime=1452786542068, timeTaken=161, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=994050048, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573084032, Process_Duration=13, Process_End_Time=2905573084045, Read_Duration=0, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:49:02,071 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786541750_0001_1 finished with state: SUCCEEDED
2016-01-14 15:49:02,071 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786541750_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:49:02,071 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786541750_0001_00_000002
2016-01-14 15:49:02,071 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786541750_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:49:02 Completed Dag: dag_1452786541750_0001_1
2016-01-14 15:49:02,072 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786541750_0001_1, dagState=SUCCEEDED
2016-01-14 15:49:02,073 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786541750_0001_1_post
2016-01-14 15:49:02,073 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:49:02,073 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:49:02,076 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sive/false_true_true_true, applicationId=application_1452786541750_0001
2016-01-14 15:49:02,077 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:49:02,077 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:49:02,077 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:49:02,077 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:49:02,077 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:49:02,077 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:49:02,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:49:02,078 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:49:02,078 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:49:02,078 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:49:02,078 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:49:02,079 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:49:02,079 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:49:02,083 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D71E4F029DAF4B01BFEBB51A56873B8E/BC85DACBA8FF43A088284AC84FC7C23F/.tez/application_1452786541750_0001
2016-01-14 15:49:02,083 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] completed in: 00:00.348
2016-01-14 15:49:02,118 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false
2016-01-14 15:49:02,118 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false
2016-01-14 15:49:02,118 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_true
2016-01-14 15:49:02,118 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_true
2016-01-14 15:49:02,118 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false
2016-01-14 15:49:02,119 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false
2016-01-14 15:49:02,119 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_true
2016-01-14 15:49:02,119 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_true
2016-01-14 15:49:02,119 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_true
2016-01-14 15:49:02,119 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_true
2016-01-14 15:49:02,119 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_true
2016-01-14 15:49:02,120 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_true
2016-01-14 15:49:02,120 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false
2016-01-14 15:49:02,120 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_true
2016-01-14 15:49:02,120 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_true
2016-01-14 15:49:02,120 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_true
2016-01-14 15:49:02,121 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false
2016-01-14 15:49:02,121 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_false
2016-01-14 15:49:02,121 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_true_false
2016-01-14 15:49:02,121 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false
2016-01-14 15:49:02,121 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false
2016-01-14 15:49:02,121 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_true_false
2016-01-14 15:49:02,122 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_true_false
2016-01-14 15:49:02,122 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_true_true