Class cascading.SortedValuesPlatformTest

4

tests

0

failures

0

ignored

1m3.35s

duration

100%

successful

Tests

Test Duration Result
testCoGroupComparatorValues 13.845s passed
testCoGroupComparatorValuesReversed 19.881s passed
testComprehensiveGroupBy 29.563s passed
testSortFails 0.056s passed

Standard output

2016-01-14 15:47:58,701 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2016-01-14 15:47:58,730 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.SortedValuesPlatformTest, with platform: hadoop2-tez
2016-01-14 15:47:58,774 INFO  tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-01-14 15:47:59,154 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-01-14 15:47:59,515 INFO  platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides: 
2016-01-14 15:47:59,729 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:47:59,731 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:47:59,852 INFO  property.AppProps (AppProps.java:getAppID(177)) - using app.id: 1D354BB1A9E5435C8B8ABC52C8A254BC
2016-01-14 15:48:00,087 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.117
2016-01-14 15:48:00,135 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.166
2016-01-14 15:48:00,137 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:48:00,138 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:48:00,140 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:00,294 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 15:48:00,465 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt
2016-01-14 15:48:00,507 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed
2016-01-14 15:48:00,781 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.4
2016-01-14 15:48:00,782 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] starting
2016-01-14 15:48:00,783 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:00,783 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:00,784 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"]
2016-01-14 15:48:00,784 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  parallel execution of steps is enabled: false
2016-01-14 15:48:00,785 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  executing total steps: 1
2016-01-14 15:48:00,785 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  allocating management threads: 1
2016-01-14 15:48:00,786 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] starting step: (1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:00,786 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [apache*ip] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:00,802 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tez session mode enabled: true
2016-01-14 15:48:00,976 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:00,985 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:00,986 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:01,032 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001 doesn't exist and is created
2016-01-14 15:48:01,093 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:01,094 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:01,095 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001
2016-01-14 15:48:01,127 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786480985_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:01,169 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:01,186 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:01,194 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-01-14 15:48:01,194 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-01-14 15:48:01,194 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:01,233 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:01,238 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:01,243 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001/localmode-log-dir
2016-01-14 15:48:01,244 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001/localmode-log-dir/history.txt.appattempt_1452786480985_0001_000000, maxErrors=10
2016-01-14 15:48:01,244 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:01,246 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786480985_0001_000000, appSubmitTime=1452786481110, launchTime=1452786481125
2016-01-14 15:48:01,257 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:01,257 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:01,263 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:01,264 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:01,295 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 15:48:01,295 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 15:48:01,295 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:01,328 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:47536
2016-01-14 15:48:01,368 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:01,395 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 15:48:01,395 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 15:48:01,395 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:01,436 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:01,436 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786480985_0001_000000, startTime=1452786481435
2016-01-14 15:48:01,436 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:01,496 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:01,514 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:01,514 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1452786480985_0001, dagName=[BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:01,714 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:01,811 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786480985_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001/localmode-log-dir/dag_1452786480985_0001_1.dot
2016-01-14 15:48:01,815 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001/localmode-log-dir/dag_1452786480985_0001_1-tez-dag.pb.txt
2016-01-14 15:48:01,921 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786480985_0001_1
2016-01-14 15:48:01,921 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:01,922 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:01 Running Dag: dag_1452786480985_0001_1
2016-01-14 15:48:01,976 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:01,980 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786480985_0001_1, submitTime=1452786481714
2016-01-14 15:48:02,008 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,016 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:02,019 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786480985_0001_1, initTime=1452786481981
2016-01-14 15:48:02,020 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786480985_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:02,019 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:02,023 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:02,023 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_STARTED]: dagID=dag_1452786480985_0001_1, startTime=1452786482023
2016-01-14 15:48:02,024 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:02,026 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786480985_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:02,026 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C0B493A9663346E5ADB005D506F37D96 : {B4B66DEC01D946AAAA58980743758C63={InputName=B4B66DEC01D946AAAA58980743758C63}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:02,026 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B4B66DEC01D946AAAA58980743758C63, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:02,027 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,032 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,032 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,034 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]. Starting root input initializers: 1
2016-01-14 15:48:02,035 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1452786480985_0001, dagName=[BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed
2016-01-14 15:48:02,039 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B4B66DEC01D946AAAA58980743758C63 on vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,049 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:02,049 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AE9CB00F172D424DAA1215E1A113C882 : {A79F425B275C462BBC16EF900023880C={InputName=A79F425B275C462BBC16EF900023880C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:02,050 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A79F425B275C462BBC16EF900023880C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:02,050 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,050 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,051 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,051 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]. Starting root input initializers: 1
2016-01-14 15:48:02,052 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:02,055 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,060 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:02,063 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B4B66DEC01D946AAAA58980743758C63 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:02,063 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A79F425B275C462BBC16EF900023880C on vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,069 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] submitted tez dag to app master: application_1452786480985_0001, with dag id: dag_1452786480985_0001_1
2016-01-14 15:48:02,066 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:02,074 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:02,110 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:02,111 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,126 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:02,133 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A79F425B275C462BBC16EF900023880C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:02,138 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:02,139 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:02,147 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,149 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,150 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=30089AFED8C24CFF9C76F292A1703C4B, vertexId=vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:02,154 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=30089AFED8C24CFF9C76F292A1703C4B, vertexId=vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,157 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 410
2016-01-14 15:48:02,160 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B4B66DEC01D946AAAA58980743758C63 on vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,162 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 15:48:02,162 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A79F425B275C462BBC16EF900023880C on vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,168 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=30089AFED8C24CFF9C76F292A1703C4B, vertexName=C93856C0D38D44EA93187859B133FBD7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:02,175 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=30089AFED8C24CFF9C76F292A1703C4B, vertexId=vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:02,180 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:02,181 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C93856C0D38D44EA93187859B133FBD7, vertexId=vertex_1452786480985_0001_1_02, initRequestedTime=1452786482052, initedTime=1452786482177, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:02,181 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:02,184 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:02,184 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B4B66DEC01D946AAAA58980743758C63=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:02,185 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] parallelism set to 1
2016-01-14 15:48:02,185 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:02,206 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786480985_0001_1_00, eventCount=1
2016-01-14 15:48:02,207 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:02,207 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C0B493A9663346E5ADB005D506F37D96, vertexId=vertex_1452786480985_0001_1_00, initRequestedTime=1452786482026, initedTime=1452786482206, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:02,207 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] with distanceFromRoot: 0
2016-01-14 15:48:02,208 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:02,208 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:02,214 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A79F425B275C462BBC16EF900023880C=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:02,214 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] parallelism set to 1
2016-01-14 15:48:02,214 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:02,216 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786480985_0001_1_01, eventCount=1
2016-01-14 15:48:02,217 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:02,217 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, vertexId=vertex_1452786480985_0001_1_01, initRequestedTime=1452786482049, initedTime=1452786482216, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:02,217 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] with distanceFromRoot: 0
2016-01-14 15:48:02,217 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:02,218 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:02,218 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,222 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:02,222 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786480985_0001_1_00, startRequestedTime=1452786482217, startedTime=1452786482217
2016-01-14 15:48:02,224 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:02,224 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:02,224 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,226 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:02,226 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786480985_0001_1_01, startRequestedTime=1452786482224, startedTime=1452786482224
2016-01-14 15:48:02,226 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:02,254 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:02,255 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_STARTED]: vertexName=C0B493A9663346E5ADB005D506F37D96, taskId=task_1452786480985_0001_1_00_000000, scheduledTime=1452786482254, launchTime=1452786482254
2016-01-14 15:48:02,255 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:02,256 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786480985_0001_1_00 for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] numStartedSources: 1 numSources: 2
2016-01-14 15:48:02,256 INFO  impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] numStartedSources: 1 numSources: 2
2016-01-14 15:48:02,258 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:02,259 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_STARTED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, taskId=task_1452786480985_0001_1_01_000000, scheduledTime=1452786482257, launchTime=1452786482257
2016-01-14 15:48:02,259 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:02,260 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786480985_0001_1_01 for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] numStartedSources: 2 numSources: 2
2016-01-14 15:48:02,260 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] with distanceFromRoot: 1
2016-01-14 15:48:02,262 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:02,263 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:02,267 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AE9CB00F172D424DAA1215E1A113C882 in vertex: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,267 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C0B493A9663346E5ADB005D506F37D96 in vertex: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,270 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C93856C0D38D44EA93187859B133FBD7 with 2 source tasks and 1 pending tasks
2016-01-14 15:48:02,270 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:02,271 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786480985_0001_1_02, startRequestedTime=1452786482263, startedTime=1452786482263
2016-01-14 15:48:02,271 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:02,274 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: C0B493A9663346E5ADB005D506F37D96, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B4B66DEC01D946AAAA58980743758C63, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:02,276 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:02,277 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: AE9CB00F172D424DAA1215E1A113C882, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A79F425B275C462BBC16EF900023880C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:02,277 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:02,277 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:02,280 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:02,312 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:02,316 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:02,316 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:02,320 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786480985_0001_00_000001
2016-01-14 15:48:02,320 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:02,321 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786480985_0001_00_000002
2016-01-14 15:48:02,321 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:02,327 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786480985_0001_00_000001
2016-01-14 15:48:02,327 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:02,331 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:02,333 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:02,334 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786480985_0001_00_000001, launchTime=1452786482332
2016-01-14 15:48:02,336 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000001
2016-01-14 15:48:02,339 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786480985_0001_1_00_000000_0] to container: [container_1452786480985_0001_00_000001]
2016-01-14 15:48:02,340 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:02,340 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786480985_0001_00_000001 given task: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,341 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000001: 1 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02 Starting to run new task attempt: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,341 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,342 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:02,342 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786480985_0001_00_000002
2016-01-14 15:48:02,342 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:02,343 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:02,344 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:02,344 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786480985_0001_1_00_000000_0] started. Is using containerId: [container_1452786480985_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:02,344 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:02,345 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:02,346 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C0B493A9663346E5ADB005D506F37D96, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0, startTime=1452786482341, containerId=container_1452786480985_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786480985_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:02,347 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:02,347 INFO  counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters:  GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-01-14 15:48:02,355 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:02,355 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:02,355 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:02,356 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:02,355 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786480985_0001_00_000002, launchTime=1452786482354
2016-01-14 15:48:02,359 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000002
2016-01-14 15:48:02,360 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786480985_0001_1_01_000000_0] to container: [container_1452786480985_0001_00_000002]
2016-01-14 15:48:02,360 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:02,361 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786480985_0001_1_01_000000_0] started. Is using containerId: [container_1452786480985_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:02,361 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:02,362 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786480985_0001_00_000002 given task: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,362 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02 Starting to run new task attempt: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,362 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0, startTime=1452786482361, containerId=container_1452786480985_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786480985_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:02,362 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,363 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:02,363 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:02,363 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:02,363 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:02,363 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:02,363 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:02,364 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:02,380 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:02,380 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:02,380 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: AE9CB00F172D424DAA1215E1A113C882, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A79F425B275C462BBC16EF900023880C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:02,380 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: C0B493A9663346E5ADB005D506F37D96, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B4B66DEC01D946AAAA58980743758C63, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:02,383 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:02,383 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:02,388 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:02,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,393 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,394 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:02,394 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,398 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,399 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B4B66DEC01D946AAAA58980743758C63, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:02,400 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:02,401 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:02,402 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:02,402 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,407 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A79F425B275C462BBC16EF900023880C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:02,407 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,409 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786480985_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:02,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C93856C0D38D44EA93187859B133FBD7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:02,410 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:02,411 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,411 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:02,409 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786480985_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:02,415 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,414 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,414 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,419 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,420 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:02,430 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:02,435 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,434 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:02,430 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:02,443 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,442 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:02,442 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:02,439 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,445 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:02,447 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,447 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,448 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,448 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,466 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C0B493A9663346E5ADB005D506F37D96, ordinal: 1
2016-01-14 15:48:02,466 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AE9CB00F172D424DAA1215E1A113C882, ordinal: 0
2016-01-14 15:48:02,468 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0B493A9663346E5ADB005D506F37D96, mem on start (mb), free: 234, total: 281, max: 672
2016-01-14 15:48:02,468 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,468 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE9CB00F172D424DAA1215E1A113C882, mem on start (mb), free: 234, total: 281, max: 672
2016-01-14 15:48:02,469 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:02,469 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:02,470 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:02,470 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:02,469 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:02,471 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:02,471 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:02,471 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:02,471 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:02,478 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:02,481 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:02,482 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:02,482 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:02,483 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:02,484 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A79F425B275C462BBC16EF900023880C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:02,484 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C93856C0D38D44EA93187859B133FBD7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:02,484 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:02,479 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:02,486 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:02,487 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:02,487 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:02,487 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:02,488 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B4B66DEC01D946AAAA58980743758C63, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:02,488 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C93856C0D38D44EA93187859B133FBD7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:02,488 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:02,489 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,489 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:02,489 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,490 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B4B66DEC01D946AAAA58980743758C63 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:02,491 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,491 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:02,491 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,491 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A79F425B275C462BBC16EF900023880C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:02,491 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:02,492 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:02,492 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:02,490 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:02,494 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:02,494 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:02,495 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,498 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,497 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:02,501 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:02,515 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@438b5662
2016-01-14 15:48:02,515 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@70d45e92
2016-01-14 15:48:02,532 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@70d45e92
2016-01-14 15:48:02,533 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:02,533 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:02,537 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@438b5662
2016-01-14 15:48:02,537 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:02,537 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:02,542 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:02,551 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:02,580 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: A79F425B275C462BBC16EF900023880C
2016-01-14 15:48:02,580 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:02,581 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:02,582 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"] streamed: true, id: B4B66DEC01D946AAAA58980743758C63
2016-01-14 15:48:02,582 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:02,583 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:02,602 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@3205d005; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@235c6976; comparator=cascading.tuple.hadoop.util.TupleComparator@80fd41d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:02,602 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1c91f372; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1b9c11eb; comparator=cascading.tuple.hadoop.util.TupleComparator@472ec00d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:02,605 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:02,606 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:02,607 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:02,608 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:02,609 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:02,609 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:02,730 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:02,731 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:02,731 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:02,731 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:02,731 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:02,739 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:02,739 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:02,742 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C0B493A9663346E5ADB005D506F37D96, all 1 inputs ready in: 00:00:00.001
2016-01-14 15:48:02,748 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:02,749 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:02,749 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:02,749 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:02,749 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:02,750 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:02,752 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:02,752 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AE9CB00F172D424DAA1215E1A113C882, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:02,757 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:02,757 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE9CB00F172D424DAA1215E1A113C882, mem on close (mb), free: 121, total: 382, max: 672
2016-01-14 15:48:02,757 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:02,757 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:02,757 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 102; bufvoid = 104857600
2016-01-14 15:48:02,758 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 15:48:02,792 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:02,792 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0B493A9663346E5ADB005D506F37D96, mem on close (mb), free: 115, total: 382, max: 672
2016-01-14 15:48:02,792 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:02,793 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:02,793 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 9061; bufvoid = 104857600
2016-01-14 15:48:02,793 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2016-01-14 15:48:05,010 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:05 Completed running task attempt: attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:05,035 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:05,036 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786480985_0001_1_01, eventCount=1
2016-01-14 15:48:05,039 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=174, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=99, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=501219328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786482752, Process_Duration=5, Process_End_Time=1452786482757, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 15:48:05,039 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:05,042 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, taskAttemptId=attempt_1452786480985_0001_1_01_000000_0, startTime=1452786482361, finishTime=1452786485037, timeTaken=2676, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=174, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=99, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=501219328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482752, Process_Duration=5, Process_End_Time=1452786482757, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:48:05,043 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:05,043 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:05,045 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:05,046 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:05,046 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000002
2016-01-14 15:48:05,047 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786480985_0001_1_01_000000_0
2016-01-14 15:48:05,049 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:05,053 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_FINISHED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, taskId=task_1452786480985_0001_1_01_000000, startTime=1452786482361, finishTime=1452786485048, timeTaken=2687, status=SUCCEEDED, successfulAttemptID=attempt_1452786480985_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=174, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=99, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=501219328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482752, Process_Duration=5, Process_End_Time=1452786482757, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:48:05,054 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:05,054 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:05,055 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] : 1
2016-01-14 15:48:05,055 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:05,069 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:05,077 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_FINISHED]: vertexName=AE9CB00F172D424DAA1215E1A113C882, vertexId=vertex_1452786480985_0001_1_01, initRequestedTime=1452786482049, initedTime=1452786482216, startRequestedTime=1452786482224, startedTime=1452786482224, finishTime=1452786485055, timeTaken=2831, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=174, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=99, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=501219328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482752, Process_Duration=5, Process_End_Time=1452786482757, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452786482361, firstTasksToStart=[ task_1452786480985_0001_1_01_000000 ], lastTaskFinishTime=1452786485037, lastTasksToFinish=[ task_1452786480985_0001_1_01_000000 ], minTaskDuration=2676, maxTaskDuration=2676, avgTaskDuration=2676.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786480985_0001_1_01_000000 ], longestDurationTasks=[ task_1452786480985_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:05,078 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:05,080 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786480985_0001_00_000002, stoppedTime=1452786485080, exitStatus=0
2016-01-14 15:48:05,081 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:05,080 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:05,082 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] attempt: attempt_1452786480985_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:05,085 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:05,085 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786480985_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:48:05,085 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:05,107 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786480985_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:05,108 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000002: 62 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:05,108 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786480985_0001_00_000002, Exiting
2016-01-14 15:48:05,109 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786480985_0001_00_000002
2016-01-14 15:48:05,109 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786480985_0001_00_000002
2016-01-14 15:48:05,111 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786480985_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:05,112 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786480985_0001_00_000002 completed successfully
2016-01-14 15:48:05,112 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:18,349 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:18 Completed running task attempt: attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:18,354 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:18,355 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786480985_0001_1_00, eventCount=1
2016-01-14 15:48:18,355 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17077, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=200, GC_TIME_MILLIS=250, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786482742, Process_Duration=50, Process_End_Time=1452786482792, Read_Duration=5, Tuples_Read=200, Tuples_Written=200, Write_Duration=13][cascading.flow.StepCounters Tuples_Read=200]]
2016-01-14 15:48:18,355 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:18,356 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:18,357 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:18,358 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000001
2016-01-14 15:48:18,359 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C0B493A9663346E5ADB005D506F37D96, taskAttemptId=attempt_1452786480985_0001_1_00_000000_0, startTime=1452786482341, finishTime=1452786498355, timeTaken=16014, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17077, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=250, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482742, Process_Duration=50, Process_End_Time=1452786482792, Read_Duration=5, Tuples_Read=200, Tuples_Written=200, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=200
2016-01-14 15:48:18,359 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:18,360 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786480985_0001_1_00_000000_0
2016-01-14 15:48:18,360 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:18,361 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:18,362 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_FINISHED]: vertexName=C0B493A9663346E5ADB005D506F37D96, taskId=task_1452786480985_0001_1_00_000000, startTime=1452786482341, finishTime=1452786498360, timeTaken=16019, status=SUCCEEDED, successfulAttemptID=attempt_1452786480985_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17077, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=250, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482742, Process_Duration=50, Process_End_Time=1452786482792, Read_Duration=5, Tuples_Read=200, Tuples_Written=200, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=200
2016-01-14 15:48:18,362 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:18,362 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] : 1
2016-01-14 15:48:18,362 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:18,365 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:18,367 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_FINISHED]: vertexName=C0B493A9663346E5ADB005D506F37D96, vertexId=vertex_1452786480985_0001_1_00, initRequestedTime=1452786482026, initedTime=1452786482206, startRequestedTime=1452786482217, startedTime=1452786482217, finishTime=1452786498364, timeTaken=16147, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17077, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=250, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786482742, Process_Duration=50, Process_End_Time=1452786482792, Read_Duration=5, Tuples_Read=200, Tuples_Written=200, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=200, vertexStats=firstTaskStartTime=1452786482341, firstTasksToStart=[ task_1452786480985_0001_1_00_000000 ], lastTaskFinishTime=1452786498355, lastTasksToFinish=[ task_1452786480985_0001_1_00_000000 ], minTaskDuration=16014, maxTaskDuration=16014, avgTaskDuration=16014.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786480985_0001_1_00_000000 ], longestDurationTasks=[ task_1452786480985_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:18,367 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:18,367 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:18,368 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] attempt: attempt_1452786480985_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:18,368 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C93856C0D38D44EA93187859B133FBD7
2016-01-14 15:48:18,368 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:18,370 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:18,370 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786480985_0001_00_000001, stoppedTime=1452786498370, exitStatus=0
2016-01-14 15:48:18,371 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:18,371 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:18,372 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:18,372 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_STARTED]: vertexName=C93856C0D38D44EA93187859B133FBD7, taskId=task_1452786480985_0001_1_02_000000, scheduledTime=1452786498372, launchTime=1452786498372
2016-01-14 15:48:18,373 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:18,373 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786480985_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:18,374 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:18,374 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:18,375 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: C93856C0D38D44EA93187859B133FBD7, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AE9CB00F172D424DAA1215E1A113C882, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C0B493A9663346E5ADB005D506F37D96, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=30089AFED8C24CFF9C76F292A1703C4B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:18,375 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:18,375 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:18,376 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786480985_0001_00_000003
2016-01-14 15:48:18,377 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:18,379 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786480985_0001_00_000003
2016-01-14 15:48:18,379 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:18,382 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:18,382 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786480985_0001_00_000003, launchTime=1452786498382
2016-01-14 15:48:18,382 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:18,383 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000003
2016-01-14 15:48:18,384 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786480985_0001_1_02_000000_0] to container: [container_1452786480985_0001_00_000003]
2016-01-14 15:48:18,384 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:18,385 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786480985_0001_1_02_000000_0] started. Is using containerId: [container_1452786480985_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 15:48:18,385 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C93856C0D38D44EA93187859B133FBD7, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0, startTime=1452786498385, containerId=container_1452786480985_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786480985_0001_00_000003/teamcity, completedLogs=
2016-01-14 15:48:18,385 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:18,385 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:18,386 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:18,386 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786480985_0001_00_000003 given task: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,386 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18 Starting to run new task attempt: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,387 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,387 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:18,387 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:18,387 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:18,387 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:18,387 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:18,387 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:18,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556] apache*ip/(1/1) ...ed/cogroupcomparereversed, VertexName: C93856C0D38D44EA93187859B133FBD7, VertexParallelism: 1, TaskAttemptID:attempt_1452786480985_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AE9CB00F172D424DAA1215E1A113C882, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=C0B493A9663346E5ADB005D506F37D96, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=30089AFED8C24CFF9C76F292A1703C4B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:18,388 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:18,388 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,388 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:18,389 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:18,389 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AE9CB00F172D424DAA1215E1A113C882, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:18,390 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C0B493A9663346E5ADB005D506F37D96, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:18,390 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:18,391 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:18,391 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:18,391 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=30089AFED8C24CFF9C76F292A1703C4B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:18,392 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:18,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 30089AFED8C24CFF9C76F292A1703C4B
2016-01-14 15:48:18,393 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,393 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:18,394 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 30089AFED8C24CFF9C76F292A1703C4B
2016-01-14 15:48:18,394 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:18,397 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,397 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786480985_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:18,397 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000001: 38 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:18,397 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786480985_0001_00_000001, Exiting
2016-01-14 15:48:18,397 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786480985_0001_00_000001
2016-01-14 15:48:18,397 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786480985_0001_00_000001
2016-01-14 15:48:18,412 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786480985_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:18,412 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786480985_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 15:48:18,412 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786480985_0001_00_000001 completed successfully
2016-01-14 15:48:18,413 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:18,414 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C93856C0D38D44EA93187859B133FBD7, ordinal: 2
2016-01-14 15:48:18,414 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C93856C0D38D44EA93187859B133FBD7, mem on start (mb), free: 124, total: 493, max: 672
2016-01-14 15:48:18,414 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:18,415 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 15:48:18,419 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:18,421 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:18,421 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,419 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:18,422 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:18,422 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,423 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:18,424 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:18,429 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:18,429 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 30089AFED8C24CFF9C76F292A1703C4B
2016-01-14 15:48:18,430 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:18,430 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:18,430 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:18,430 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:18,430 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:18,430 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:18,431 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 15:48:18,431 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 15:48:18,431 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 30089AFED8C24CFF9C76F292A1703C4B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:18,431 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AE9CB00F172D424DAA1215E1A113C882, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:48:18,431 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C0B493A9663346E5ADB005D506F37D96, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:48:18,432 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:18,432 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AE9CB00F172D424DAA1215E1A113C882 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:18,432 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,433 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,434 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C0B493A9663346E5ADB005D506F37D96 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:18,434 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 15:48:18,435 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 15:48:18,440 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:18,440 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:18,441 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:18,441 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:18,445 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C0B493A9663346E5ADB005D506F37D96 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:18,445 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AE9CB00F172D424DAA1215E1A113C882 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:18,448 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:48:18,448 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:18,448 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:48:18,448 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:18,452 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:18,452 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:18,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,456 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:18,457 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:18,457 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,459 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786480985_0001_1_01_000000_0_10006, runDuration: 2598164, ]
2016-01-14 15:48:18,462 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786480985_0001_1_00_000000_0_10004, runDuration: 15931612, ]
2016-01-14 15:48:18,465 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2016-01-14 15:48:18,465 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786480985_0001_1_01_000000_0_10006] to DISK_DIRECT, CompressedSize=142, DecompressedSize=138,EndTime=1452786498465, TimeTaken=2, Rate=0.07 MB/s
2016-01-14 15:48:18,465 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786480985_0001_1_00_000000_0_10004] to DISK_DIRECT, CompressedSize=8931, DecompressedSize=8927,EndTime=1452786498465, TimeTaken=2, Rate=4.26 MB/s
2016-01-14 15:48:18,466 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,466 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,466 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s)
2016-01-14 15:48:18,467 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C0B493A9663346E5ADB005D506F37D96] #2 in 4ms
2016-01-14 15:48:18,467 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C0B493A9663346E5ADB005D506F37D96] #2, status:false, isInterrupted:false
2016-01-14 15:48:18,468 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:18,468 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:18,469 WARN  util.TupleHasher (TupleHasher.java:merge(73)) - two unequal Hasher instances for the same key field position found: cascading.platform.hadoop.TestLongComparator@7e308c04, and: cascading.platform.hadoop.TestLongComparator@25389b55
2016-01-14 15:48:18,470 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] streamed: true, id: 4B42A92D7FD54B7C99C3C46A0070B663
2016-01-14 15:48:18,470 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"], id: 30089AFED8C24CFF9C76F292A1703C4B
2016-01-14 15:48:18,470 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed"]
2016-01-14 15:48:18,470 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 8931 bytes from disk
2016-01-14 15:48:18,471 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AE9CB00F172D424DAA1215E1A113C882] #1 in 8ms
2016-01-14 15:48:18,471 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:18,471 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AE9CB00F172D424DAA1215E1A113C882] #1, status:false, isInterrupted:false
2016-01-14 15:48:18,472 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:18,473 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 142 bytes from disk
2016-01-14 15:48:18,473 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:18,473 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 4B42A92D7FD54B7C99C3C46A0070B663, for 2 inputs
2016-01-14 15:48:18,474 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] 4B42A92D7FD54B7C99C3C46A0070B663, for 2 inputs
2016-01-14 15:48:18,478 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:18,478 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:18,481 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:48:18,481 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:48:18,487 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 8917 bytes
2016-01-14 15:48:18,487 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,488 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 128 bytes
2016-01-14 15:48:18,488 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,489 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:18,489 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C93856C0D38D44EA93187859B133FBD7, all 2 inputs ready in: 00:00:00.000
2016-01-14 15:48:18,490 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:18,494 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@76fc3212; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@273b78c5
2016-01-14 15:48:18,496 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3175e1a2; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@32306c0c
2016-01-14 15:48:18,526 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:18,526 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:18,526 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,527 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786480985_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 15:48:18,528 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,528 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C93856C0D38D44EA93187859B133FBD7, mem on close (mb), free: 112, total: 493, max: 672
2016-01-14 15:48:18,528 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AE9CB00F172D424DAA1215E1A113C882
2016-01-14 15:48:18,529 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:18,529 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:18,530 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:18,530 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:18,530 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:18,531 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C0B493A9663346E5ADB005D506F37D96
2016-01-14 15:48:18,531 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:18,532 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:18,532 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:18,532 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:18,532 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:18 Completed running task attempt: attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,534 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:18,536 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:18,537 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786498490, Process_Duration=38, Process_End_Time=1452786498528, Read_Duration=13, Tuples_Read=124, Tuples_Written=199, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=199]]
2016-01-14 15:48:18,538 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C93856C0D38D44EA93187859B133FBD7, taskAttemptId=attempt_1452786480985_0001_1_02_000000_0, startTime=1452786498385, finishTime=1452786498536, timeTaken=151, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786498490, Process_Duration=38, Process_End_Time=1452786498528, Read_Duration=13, Tuples_Read=124, Tuples_Written=199, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=199
2016-01-14 15:48:18,538 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786480985_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:18,538 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:18,539 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:18,540 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:18,538 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786480985_0001_1_02_000000_0
2016-01-14 15:48:18,541 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:18,541 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786480985_0001_00_000003
2016-01-14 15:48:18,543 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:TASK_FINISHED]: vertexName=C93856C0D38D44EA93187859B133FBD7, taskId=task_1452786480985_0001_1_02_000000, startTime=1452786498385, finishTime=1452786498540, timeTaken=155, status=SUCCEEDED, successfulAttemptID=attempt_1452786480985_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786498490, Process_Duration=38, Process_End_Time=1452786498528, Read_Duration=13, Tuples_Read=124, Tuples_Written=199, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=199
2016-01-14 15:48:18,543 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786480985_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:18,543 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] : 1
2016-01-14 15:48:18,543 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:18,544 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:18,545 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:VERTEX_FINISHED]: vertexName=C93856C0D38D44EA93187859B133FBD7, vertexId=vertex_1452786480985_0001_1_02, initRequestedTime=1452786482052, initedTime=1452786482177, startRequestedTime=1452786482263, startedTime=1452786482263, finishTime=1452786498543, timeTaken=16280, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=516947968, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786498490, Process_Duration=38, Process_End_Time=1452786498528, Read_Duration=13, Tuples_Read=124, Tuples_Written=199, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=199, vertexStats=firstTaskStartTime=1452786498385, firstTasksToStart=[ task_1452786480985_0001_1_02_000000 ], lastTaskFinishTime=1452786498536, lastTasksToFinish=[ task_1452786480985_0001_1_02_000000 ], minTaskDuration=151, maxTaskDuration=151, avgTaskDuration=151.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786480985_0001_1_02_000000 ], longestDurationTasks=[ task_1452786480985_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:18,546 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:18,546 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:18,547 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786480985_0001_00_000003, stoppedTime=1452786498547, exitStatus=0
2016-01-14 15:48:18,547 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:18,547 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:18,547 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:18,548 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786480985_0001_1
2016-01-14 15:48:18,548 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786480985_0001_1
2016-01-14 15:48:18,549 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 30089AFED8C24CFF9C76F292A1703C4B for vertex: vertex_1452786480985_0001_1_02 [C93856C0D38D44EA93187859B133FBD7]
2016-01-14 15:48:18,552 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786480985_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 15:48:18,552 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786480985_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:18,553 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786480985_0001_00_000003, Exiting
2016-01-14 15:48:18,553 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786480985_0001_00_000003
2016-01-14 15:48:18,553 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786480985_0001_00_000003
2016-01-14 15:48:18,553 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786480985_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:18,554 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786480985_0001_00_000003 completed successfully
2016-01-14 15:48:18,560 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786480985_0001_1_00 [C0B493A9663346E5ADB005D506F37D96]
2016-01-14 15:48:18,561 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786480985_0001_1_01 [AE9CB00F172D424DAA1215E1A113C882]
2016-01-14 15:48:18,567 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786480985_0001_1, queueSize=0
2016-01-14 15:48:18,568 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:18,568 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786480985_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786480985_0001_1, startTime=1452786482023, finishTime=1452786498561, timeTaken=16538, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=83619, FILE_BYTES_WRITTEN=23910, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=440, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=349, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1535115264, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=419, OUTPUT_BYTES=9163, OUTPUT_BYTES_WITH_OVERHEAD=9065, OUTPUT_BYTES_PHYSICAL=9073, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=47, FIRST_EVENT_RECEIVED=21, LAST_EVENT_RECEIVED=22, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358359463984, Process_Duration=93, Process_End_Time=4358359464077, Read_Duration=21, Tuples_Read=344, Tuples_Written=419, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=220, Tuples_Written=199
2016-01-14 15:48:18,569 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786480985_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:18,569 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786480985_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:18,570 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786480985_0001_00_000003
2016-01-14 15:48:18,570 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786480985_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:18 Completed Dag: dag_1452786480985_0001_1
2016-01-14 15:48:18,572 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786480985_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:18,572 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786480985_0001_1_post
2016-01-14 15:48:18,573 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:18,573 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:18,578 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ed/cogroupcomparereversed, applicationId=application_1452786480985_0001
2016-01-14 15:48:18,581 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:18,581 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:18,581 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:18,581 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:18,581 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:18,582 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:18,585 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:18,585 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:18,585 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:18,585 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:18,585 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:18,586 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:18,586 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:18,591 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BA6800A088BB4B09974D0D181E075F47/CC031C9BE71A4C37A2C84E4BAE11D556/.tez/application_1452786480985_0001
2016-01-14 15:48:18,591 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  completed in: 00:17.809
2016-01-14 15:48:18,653 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValuesReversed/cogroupcomparereversed
2016-01-14 15:48:18,685 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:18,686 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:18,758 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-01-14 15:48:18,803 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.088
2016-01-14 15:48:18,804 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:48:18,805 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 15:48:18,806 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:18,821 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 15:48:18,838 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt
2016-01-14 15:48:18,856 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward
2016-01-14 15:48:18,942 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] starting
2016-01-14 15:48:18,943 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:18,943 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:18,943 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"]
2016-01-14 15:48:18,943 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  parallel execution of steps is enabled: false
2016-01-14 15:48:18,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  executing total steps: 1
2016-01-14 15:48:18,944 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  allocating management threads: 1
2016-01-14 15:48:18,945 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] starting step: (1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:18,947 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [apache*ip] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:18,956 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] tez session mode enabled: true
2016-01-14 15:48:18,962 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:18,963 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:18,963 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:18,973 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001 doesn't exist and is created
2016-01-14 15:48:18,992 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:18,993 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:18,994 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001
2016-01-14 15:48:18,998 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786498963_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:19,015 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:19,017 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:19,020 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:19,021 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:19,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001/localmode-log-dir
2016-01-14 15:48:19,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001/localmode-log-dir/history.txt.appattempt_1452786498963_0001_000000, maxErrors=10
2016-01-14 15:48:19,021 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:19,022 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786498963_0001_000000, appSubmitTime=1452786498997, launchTime=1452786498997
2016-01-14 15:48:19,029 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:19,034 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:19,036 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:19,036 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:45252
2016-01-14 15:48:19,037 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:19,040 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:19,042 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786498963_0001_000000, startTime=1452786499042
2016-01-14 15:48:19,042 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:19,042 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:19,093 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:19,093 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:19,093 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1452786498963_0001, dagName=[8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:19,110 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:19,113 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786498963_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001/localmode-log-dir/dag_1452786498963_0001_1.dot
2016-01-14 15:48:19,114 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001/localmode-log-dir/dag_1452786498963_0001_1-tez-dag.pb.txt
2016-01-14 15:48:19,146 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786498963_0001_1
2016-01-14 15:48:19,146 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:19,147 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:19 Running Dag: dag_1452786498963_0001_1
2016-01-14 15:48:19,164 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786498963_0001_1, submitTime=1452786499110
2016-01-14 15:48:19,164 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:19,166 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,166 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:19,166 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786498963_0001_1, initTime=1452786499164
2016-01-14 15:48:19,167 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786498963_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:19,166 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:19,168 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_STARTED]: dagID=dag_1452786498963_0001_1, startTime=1452786499168
2016-01-14 15:48:19,169 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:19,169 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:19,169 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786498963_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:19,170 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E2E2D7DC0F3041E7ABC17AC0FA602344 : {3EA1543B792642C5878D8BFD49565491={InputName=3EA1543B792642C5878D8BFD49565491}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:19,170 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3EA1543B792642C5878D8BFD49565491, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:19,170 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,171 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,171 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,171 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1452786498963_0001, dagName=[8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward
2016-01-14 15:48:19,171 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]. Starting root input initializers: 1
2016-01-14 15:48:19,172 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:19,172 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3EA1543B792642C5878D8BFD49565491 on vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,172 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D262ACA9B2A94E9E8430EB2120B6B3A9 : {2E0F3C807AC941DB9890A121B97B8186={InputName=2E0F3C807AC941DB9890A121B97B8186}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:19,172 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2E0F3C807AC941DB9890A121B97B8186, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:19,172 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,173 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,173 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,173 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]. Starting root input initializers: 1
2016-01-14 15:48:19,174 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:19,174 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,174 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2E0F3C807AC941DB9890A121B97B8186 on vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,176 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:19,177 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3EA1543B792642C5878D8BFD49565491 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:19,179 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:19,179 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:19,182 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 410
2016-01-14 15:48:19,185 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3EA1543B792642C5878D8BFD49565491 on vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,183 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:19,186 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,186 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,187 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,187 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B71CA62645984090B1A2804D20C09E5D, vertexId=vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:19,188 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B71CA62645984090B1A2804D20C09E5D, vertexId=vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,191 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:19,191 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2E0F3C807AC941DB9890A121B97B8186 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:19,194 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:19,195 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:19,193 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B71CA62645984090B1A2804D20C09E5D, vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:19,198 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B71CA62645984090B1A2804D20C09E5D, vertexId=vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:19,200 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip] submitted tez dag to app master: application_1452786498963_0001, with dag id: dag_1452786498963_0001_1
2016-01-14 15:48:19,201 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, vertexId=vertex_1452786498963_0001_1_02, initRequestedTime=1452786499174, initedTime=1452786499200, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:19,201 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:19,202 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 15:48:19,203 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:19,204 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:19,204 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3EA1543B792642C5878D8BFD49565491=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:19,204 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] parallelism set to 1
2016-01-14 15:48:19,204 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:19,203 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2E0F3C807AC941DB9890A121B97B8186 on vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,206 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786498963_0001_1_00, eventCount=1
2016-01-14 15:48:19,207 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:19,208 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, vertexId=vertex_1452786498963_0001_1_00, initRequestedTime=1452786499170, initedTime=1452786499207, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:19,208 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] with distanceFromRoot: 0
2016-01-14 15:48:19,208 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:19,209 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:19,209 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2E0F3C807AC941DB9890A121B97B8186=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:19,211 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] parallelism set to 1
2016-01-14 15:48:19,211 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:19,212 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786498963_0001_1_01, eventCount=1
2016-01-14 15:48:19,213 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, vertexId=vertex_1452786498963_0001_1_01, initRequestedTime=1452786499172, initedTime=1452786499212, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:19,213 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] with distanceFromRoot: 0
2016-01-14 15:48:19,213 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:19,213 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:19,214 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:19,214 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,215 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786498963_0001_1_00, startRequestedTime=1452786499214, startedTime=1452786499214
2016-01-14 15:48:19,215 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:19,216 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:19,216 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:19,216 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,217 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:19,217 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786498963_0001_1_01, startRequestedTime=1452786499216, startedTime=1452786499216
2016-01-14 15:48:19,217 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:19,218 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:19,219 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_STARTED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, taskId=task_1452786498963_0001_1_00_000000, scheduledTime=1452786499218, launchTime=1452786499218
2016-01-14 15:48:19,219 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:19,219 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786498963_0001_1_00 for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] numStartedSources: 1 numSources: 2
2016-01-14 15:48:19,220 INFO  impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] numStartedSources: 1 numSources: 2
2016-01-14 15:48:19,220 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:19,221 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_STARTED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, taskId=task_1452786498963_0001_1_01_000000, scheduledTime=1452786499220, launchTime=1452786499220
2016-01-14 15:48:19,221 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:19,221 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786498963_0001_1_01 for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] numStartedSources: 2 numSources: 2
2016-01-14 15:48:19,221 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] with distanceFromRoot: 1
2016-01-14 15:48:19,221 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:19,222 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:19,222 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D262ACA9B2A94E9E8430EB2120B6B3A9 in vertex: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,222 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E2E2D7DC0F3041E7ABC17AC0FA602344 in vertex: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,222 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A0C94E7A8AEA45F0A8819AD7A58541D3 with 2 source tasks and 1 pending tasks
2016-01-14 15:48:19,223 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:19,223 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786498963_0001_1_02, startRequestedTime=1452786499222, startedTime=1452786499222
2016-01-14 15:48:19,223 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:19,223 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: E2E2D7DC0F3041E7ABC17AC0FA602344, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3EA1543B792642C5878D8BFD49565491, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:19,224 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:19,224 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: D262ACA9B2A94E9E8430EB2120B6B3A9, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2E0F3C807AC941DB9890A121B97B8186, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:19,224 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:19,225 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:19,225 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:19,225 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:19,226 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:19,226 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:19,226 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786498963_0001_00_000001
2016-01-14 15:48:19,226 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:19,227 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786498963_0001_00_000002
2016-01-14 15:48:19,227 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:19,229 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786498963_0001_00_000001
2016-01-14 15:48:19,229 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:19,230 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:19,231 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:19,231 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786498963_0001_00_000001, launchTime=1452786499230
2016-01-14 15:48:19,232 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000001
2016-01-14 15:48:19,233 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786498963_0001_1_00_000000_0] to container: [container_1452786498963_0001_00_000001]
2016-01-14 15:48:19,233 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:19,234 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786498963_0001_00_000001 given task: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,234 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786498963_0001_1_00_000000_0] started. Is using containerId: [container_1452786498963_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:19,234 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0, startTime=1452786499234, containerId=container_1452786498963_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786498963_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:19,234 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:19,235 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786498963_0001_00_000002
2016-01-14 15:48:19,235 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:19,235 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:19,234 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,235 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:19,235 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:19,236 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:19,236 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786498963_0001_00_000002, launchTime=1452786499235
2016-01-14 15:48:19,237 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000002
2016-01-14 15:48:19 Starting to run new task attempt: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,239 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,240 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:19,240 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:19,240 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:19,240 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:19,240 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:19,242 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:19,242 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: E2E2D7DC0F3041E7ABC17AC0FA602344, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3EA1543B792642C5878D8BFD49565491, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:19,242 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:19,241 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786498963_0001_1_01_000000_0] to container: [container_1452786498963_0001_00_000002]
2016-01-14 15:48:19,243 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,243 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:19,244 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786498963_0001_00_000002 given task: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,244 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,244 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786498963_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:19 Starting to run new task attempt: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,244 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:19,245 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,247 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:19,247 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:19,247 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:19,247 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:19,247 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:19,247 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:19,247 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: D262ACA9B2A94E9E8430EB2120B6B3A9, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2E0F3C807AC941DB9890A121B97B8186, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:19,245 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786498963_0001_1_01_000000_0] started. Is using containerId: [container_1452786498963_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:19,249 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0, startTime=1452786499244, containerId=container_1452786498963_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786498963_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:19,249 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:19,249 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:19,249 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:19,250 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:19,251 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,253 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,253 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:19,253 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,254 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3EA1543B792642C5878D8BFD49565491, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:19,254 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,254 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:19,257 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786498963_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:19,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2E0F3C807AC941DB9890A121B97B8186, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:19,257 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:19,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:19,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:19,259 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,261 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:19,261 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,262 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,258 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:19,263 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,264 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,260 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:19,264 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:19,264 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,266 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:19,272 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:19,272 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,272 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,275 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:19,275 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,275 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,275 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:19,277 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:19,278 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,279 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,288 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D262ACA9B2A94E9E8430EB2120B6B3A9, ordinal: 0
2016-01-14 15:48:19,288 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D262ACA9B2A94E9E8430EB2120B6B3A9, mem on start (mb), free: 203, total: 493, max: 672
2016-01-14 15:48:19,288 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:19,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:19,289 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:19,289 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:19,289 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:19,290 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:19,290 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:19,290 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:19,291 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:19,291 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2E0F3C807AC941DB9890A121B97B8186, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:19,291 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A0C94E7A8AEA45F0A8819AD7A58541D3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:19,292 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:19,292 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2E0F3C807AC941DB9890A121B97B8186 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:19,292 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,292 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:19,292 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,293 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:19,294 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:19,295 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:19,295 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,297 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:19,300 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7a223f56
2016-01-14 15:48:19,301 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E2E2D7DC0F3041E7ABC17AC0FA602344, ordinal: 1
2016-01-14 15:48:19,302 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E2E2D7DC0F3041E7ABC17AC0FA602344, mem on start (mb), free: 201, total: 493, max: 672
2016-01-14 15:48:19,302 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:19,302 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:19,303 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:19,305 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:19,306 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:19,306 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:19,307 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:19,308 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:19,309 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:19,309 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:19,310 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3EA1543B792642C5878D8BFD49565491, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:19,311 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A0C94E7A8AEA45F0A8819AD7A58541D3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:19,311 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:19,305 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7a223f56
2016-01-14 15:48:19,305 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:19,312 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:19,314 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,313 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 2E0F3C807AC941DB9890A121B97B8186
2016-01-14 15:48:19,314 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3EA1543B792642C5878D8BFD49565491 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:19,314 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:19,315 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,314 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:19,315 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:19,316 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:19,317 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:19,317 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:19,317 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:19,318 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1031a99a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2c75c5c4; comparator=cascading.tuple.hadoop.util.TupleComparator@680adda9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:19,319 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:19,320 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@516873e1
2016-01-14 15:48:19,321 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@516873e1
2016-01-14 15:48:19,319 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,319 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:19,322 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:19,323 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:19,323 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:19,323 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:19,337 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:19,338 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:19,338 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:19,339 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:19,339 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:19,340 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:19,339 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"] streamed: true, id: 3EA1543B792642C5878D8BFD49565491
2016-01-14 15:48:19,340 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']], id: E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:19,340 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 15:48:19,341 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:19,341 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:19,341 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D262ACA9B2A94E9E8430EB2120B6B3A9, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:19,345 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1ba90cc; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7f7841f6; comparator=cascading.tuple.hadoop.util.TupleComparator@3032bedb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:19,345 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:19,346 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:19,346 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:19,346 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D262ACA9B2A94E9E8430EB2120B6B3A9, mem on close (mb), free: 99, total: 493, max: 672
2016-01-14 15:48:19,347 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:19,347 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:19,347 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 102; bufvoid = 104857600
2016-01-14 15:48:19,347 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 15:48:19,348 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:19,823 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:19,823 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:19,823 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:19,823 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:19,823 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:19,832 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/apache.200.txt"]
2016-01-14 15:48:19,832 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:19,832 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E2E2D7DC0F3041E7ABC17AC0FA602344, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:19,860 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:19,861 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E2E2D7DC0F3041E7ABC17AC0FA602344, mem on close (mb), free: 292, total: 520, max: 672
2016-01-14 15:48:19,861 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:19,861 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:19,861 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 9061; bufvoid = 104857600
2016-01-14 15:48:19,861 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2016-01-14 15:48:21,026 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:21 Completed running task attempt: attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:21,031 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:21,032 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786498963_0001_1_01, eventCount=1
2016-01-14 15:48:21,032 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:21,033 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, taskAttemptId=attempt_1452786498963_0001_1_01_000000_0, startTime=1452786499244, finishTime=1452786501032, timeTaken=1788, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=694, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=393, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=545783808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499342, Process_Duration=4, Process_End_Time=1452786499346, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:48:21,033 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:21,033 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786498963_0001_1_01_000000_0
2016-01-14 15:48:21,034 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_FINISHED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, taskId=task_1452786498963_0001_1_01_000000, startTime=1452786499244, finishTime=1452786501034, timeTaken=1790, status=SUCCEEDED, successfulAttemptID=attempt_1452786498963_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=694, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=393, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=545783808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499342, Process_Duration=4, Process_End_Time=1452786499346, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 15:48:21,034 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:21,034 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] : 1
2016-01-14 15:48:21,034 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:21,035 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:21,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=694, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=393, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=545783808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786499342, Process_Duration=4, Process_End_Time=1452786499346, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 15:48:21,036 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:21,043 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:21,043 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:21,041 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_FINISHED]: vertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, vertexId=vertex_1452786498963_0001_1_01, initRequestedTime=1452786499172, initedTime=1452786499212, startRequestedTime=1452786499216, startedTime=1452786499216, finishTime=1452786501034, timeTaken=1818, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37225, FILE_BYTES_WRITTEN=694, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=393, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=545783808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=102, OUTPUT_BYTES_WITH_OVERHEAD=138, OUTPUT_BYTES_PHYSICAL=142, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499342, Process_Duration=4, Process_End_Time=1452786499346, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452786499244, firstTasksToStart=[ task_1452786498963_0001_1_01_000000 ], lastTaskFinishTime=1452786501032, lastTasksToFinish=[ task_1452786498963_0001_1_01_000000 ], minTaskDuration=1788, maxTaskDuration=1788, avgTaskDuration=1788.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786498963_0001_1_01_000000 ], longestDurationTasks=[ task_1452786498963_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:21,044 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:21,044 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] attempt: attempt_1452786498963_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:21,044 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:21,045 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000002
2016-01-14 15:48:21,044 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:21,047 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786498963_0001_00_000002, stoppedTime=1452786501045, exitStatus=0
2016-01-14 15:48:21,047 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:21,047 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:21,047 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786498963_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 15:48:21,047 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:21,047 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:21,048 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786498963_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:21,048 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:21,048 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786498963_0001_00_000002, Exiting
2016-01-14 15:48:21,048 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786498963_0001_00_000002
2016-01-14 15:48:21,049 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786498963_0001_00_000002
2016-01-14 15:48:21,052 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786498963_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:21,052 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786498963_0001_00_000002 completed successfully
2016-01-14 15:48:21,053 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:23,583 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:23,583 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:32,291 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:32 Completed running task attempt: attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:32,294 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:32,295 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786498963_0001_1_00, eventCount=1
2016-01-14 15:48:32,295 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17750, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=200, GC_TIME_MILLIS=534, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786499833, Process_Duration=27, Process_End_Time=1452786499860, Read_Duration=2, Tuples_Read=200, Tuples_Written=200, Write_Duration=9][cascading.flow.StepCounters Tuples_Read=200]]
2016-01-14 15:48:32,295 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:32,295 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:32,296 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,296 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, taskAttemptId=attempt_1452786498963_0001_1_00_000000_0, startTime=1452786499234, finishTime=1452786512295, timeTaken=13061, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17750, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=534, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499833, Process_Duration=27, Process_End_Time=1452786499860, Read_Duration=2, Tuples_Read=200, Tuples_Written=200, Write_Duration=9, cascading.flow.StepCounters, Tuples_Read=200
2016-01-14 15:48:32,297 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000001
2016-01-14 15:48:32,297 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:32,297 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786498963_0001_1_00_000000_0
2016-01-14 15:48:32,297 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:32,298 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:32,298 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_FINISHED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, taskId=task_1452786498963_0001_1_00_000000, startTime=1452786499234, finishTime=1452786512298, timeTaken=13064, status=SUCCEEDED, successfulAttemptID=attempt_1452786498963_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17750, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=534, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499833, Process_Duration=27, Process_End_Time=1452786499860, Read_Duration=2, Tuples_Read=200, Tuples_Written=200, Write_Duration=9, cascading.flow.StepCounters, Tuples_Read=200
2016-01-14 15:48:32,298 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:32,299 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] : 1
2016-01-14 15:48:32,299 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:32,300 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:32,300 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_FINISHED]: vertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, vertexId=vertex_1452786498963_0001_1_00, initRequestedTime=1452786499170, initedTime=1452786499207, startRequestedTime=1452786499214, startedTime=1452786499214, finishTime=1452786512299, timeTaken=13085, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=37257, FILE_BYTES_WRITTEN=17750, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=200, GC_TIME_MILLIS=534, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=200, OUTPUT_BYTES=9061, OUTPUT_BYTES_WITH_OVERHEAD=8927, OUTPUT_BYTES_PHYSICAL=8931, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786499833, Process_Duration=27, Process_End_Time=1452786499860, Read_Duration=2, Tuples_Read=200, Tuples_Written=200, Write_Duration=9, cascading.flow.StepCounters, Tuples_Read=200, vertexStats=firstTaskStartTime=1452786499234, firstTasksToStart=[ task_1452786498963_0001_1_00_000000 ], lastTaskFinishTime=1452786512295, lastTasksToFinish=[ task_1452786498963_0001_1_00_000000 ], minTaskDuration=13061, maxTaskDuration=13061, avgTaskDuration=13061.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786498963_0001_1_00_000000 ], longestDurationTasks=[ task_1452786498963_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:32,301 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:32,301 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:32,301 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786498963_0001_00_000001, stoppedTime=1452786512301, exitStatus=0
2016-01-14 15:48:32,301 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:32,302 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] attempt: attempt_1452786498963_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:32,302 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A0C94E7A8AEA45F0A8819AD7A58541D3
2016-01-14 15:48:32,302 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:32,303 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:32,303 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786498963_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:32,303 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:32,303 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_STARTED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, taskId=task_1452786498963_0001_1_02_000000, scheduledTime=1452786512303, launchTime=1452786512303
2016-01-14 15:48:32,304 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:32,304 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:32,304 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:32,305 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:32,305 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: A0C94E7A8AEA45F0A8819AD7A58541D3, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B71CA62645984090B1A2804D20C09E5D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:32,305 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:32,306 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:32,306 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786498963_0001_00_000003
2016-01-14 15:48:32,307 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:32,307 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786498963_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:32,307 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:32,308 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786498963_0001_00_000001, Exiting
2016-01-14 15:48:32,308 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786498963_0001_00_000001
2016-01-14 15:48:32,308 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786498963_0001_00_000001
2016-01-14 15:48:32,308 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786498963_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:32,308 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786498963_0001_00_000001 completed successfully
2016-01-14 15:48:32,309 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:32,310 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786498963_0001_00_000003
2016-01-14 15:48:32,311 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:32,313 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:32,314 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:32,314 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786498963_0001_00_000003, launchTime=1452786512313
2016-01-14 15:48:32,315 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000003
2016-01-14 15:48:32,315 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786498963_0001_1_02_000000_0] to container: [container_1452786498963_0001_00_000003]
2016-01-14 15:48:32,315 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:32,316 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786498963_0001_00_000003 given task: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,316 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32 Starting to run new task attempt: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,316 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786498963_0001_1_02_000000_0] started. Is using containerId: [container_1452786498963_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 15:48:32,316 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0, startTime=1452786512316, containerId=container_1452786498963_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786498963_0001_00_000003/teamcity, completedLogs=
2016-01-14 15:48:32,316 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:32,317 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:32,316 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,317 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:32,317 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,317 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:32,318 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:32,318 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:32,318 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:32,318 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:32,319 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8] apache*ip/(1/1) ...ues/cogroupcompareforward, VertexName: A0C94E7A8AEA45F0A8819AD7A58541D3, VertexParallelism: 1, TaskAttemptID:attempt_1452786498963_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B71CA62645984090B1A2804D20C09E5D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:32,319 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:32,319 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,320 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:32,320 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,322 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786498963_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 15:48:32,323 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E2E2D7DC0F3041E7ABC17AC0FA602344, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:32,323 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D262ACA9B2A94E9E8430EB2120B6B3A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:32,323 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:32,323 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,324 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,325 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:32,325 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:32,325 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,326 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,323 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B71CA62645984090B1A2804D20C09E5D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:32,326 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:32,326 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:32,326 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B71CA62645984090B1A2804D20C09E5D
2016-01-14 15:48:32,326 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:32,327 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B71CA62645984090B1A2804D20C09E5D
2016-01-14 15:48:32,324 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:32,331 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,332 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:32,332 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:32,334 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,334 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,337 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A0C94E7A8AEA45F0A8819AD7A58541D3, ordinal: 2
2016-01-14 15:48:32,337 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0C94E7A8AEA45F0A8819AD7A58541D3, mem on start (mb), free: 254, total: 521, max: 672
2016-01-14 15:48:32,337 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,337 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 15:48:32,338 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:32,338 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:32,340 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:32,340 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B71CA62645984090B1A2804D20C09E5D
2016-01-14 15:48:32,340 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:32,340 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:32,341 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:32,341 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:32,341 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:32,341 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:32,341 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 15:48:32,342 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 15:48:32,342 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E2E2D7DC0F3041E7ABC17AC0FA602344, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:48:32,342 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B71CA62645984090B1A2804D20C09E5D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:32,342 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D262ACA9B2A94E9E8430EB2120B6B3A9, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 15:48:32,342 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:32,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E2E2D7DC0F3041E7ABC17AC0FA602344 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:32,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,343 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D262ACA9B2A94E9E8430EB2120B6B3A9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:32,346 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:32,346 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:32,347 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:32,347 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:32,347 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D262ACA9B2A94E9E8430EB2120B6B3A9 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:32,347 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:48:32,347 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:32,348 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E2E2D7DC0F3041E7ABC17AC0FA602344 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:32,348 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 15:48:32,348 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:32,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 15:48:32,349 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 15:48:32,348 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:32,349 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:32,350 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,351 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,351 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:32,354 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:32,354 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,354 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786498963_0001_1_01_000000_0_10016, runDuration: 1768849, ]
2016-01-14 15:48:32,355 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786498963_0001_1_00_000000_0_10014, runDuration: 13034431, ]
2016-01-14 15:48:32,356 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786498963_0001_1_00_000000_0_10014] to DISK_DIRECT, CompressedSize=8931, DecompressedSize=8927,EndTime=1452786512356, TimeTaken=1, Rate=8.52 MB/s
2016-01-14 15:48:32,356 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,357 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.01 MB/s)
2016-01-14 15:48:32,358 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E2E2D7DC0F3041E7ABC17AC0FA602344] #3 in 3ms
2016-01-14 15:48:32,359 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E2E2D7DC0F3041E7ABC17AC0FA602344] #3, status:false, isInterrupted:false
2016-01-14 15:48:32,361 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2016-01-14 15:48:32,361 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:32,362 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786498963_0001_1_01_000000_0_10016] to DISK_DIRECT, CompressedSize=142, DecompressedSize=138,EndTime=1452786512362, TimeTaken=4, Rate=0.03 MB/s
2016-01-14 15:48:32,363 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,362 WARN  util.TupleHasher (TupleHasher.java:merge(73)) - two unequal Hasher instances for the same key field position found: cascading.platform.hadoop.TestLongComparator@42d1ff2e, and: cascading.platform.hadoop.TestLongComparator@68c84747
2016-01-14 15:48:32,364 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] streamed: true, id: E7AFE3E8781F465AA8FDCFF0ADECFA06
2016-01-14 15:48:32,364 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"], id: B71CA62645984090B1A2804D20C09E5D
2016-01-14 15:48:32,364 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:32,365 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D262ACA9B2A94E9E8430EB2120B6B3A9] #4 in 7ms
2016-01-14 15:48:32,363 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 8931 bytes from disk
2016-01-14 15:48:32,365 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:32,365 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:32,364 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward"]
2016-01-14 15:48:32,366 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D262ACA9B2A94E9E8430EB2120B6B3A9] #4, status:false, isInterrupted:false
2016-01-14 15:48:32,366 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] E7AFE3E8781F465AA8FDCFF0ADECFA06, for 2 inputs
2016-01-14 15:48:32,367 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 8918 bytes
2016-01-14 15:48:32,367 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,366 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:32,367 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']] E7AFE3E8781F465AA8FDCFF0ADECFA06, for 2 inputs
2016-01-14 15:48:32,368 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:32,369 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 142 bytes from disk
2016-01-14 15:48:32,369 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:32,369 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:32,369 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:48:32,370 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 15:48:32,370 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 129 bytes
2016-01-14 15:48:32,370 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,371 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:32,371 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A0C94E7A8AEA45F0A8819AD7A58541D3, all 2 inputs ready in: 00:00:00.000
2016-01-14 15:48:32,374 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7737d902; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@73884f73
2016-01-14 15:48:32,376 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@27e675fc; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@b4c07db
2016-01-14 15:48:32,399 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:32,400 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:32,400 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,401 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786498963_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 15:48:32,402 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,402 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0C94E7A8AEA45F0A8819AD7A58541D3, mem on close (mb), free: 244, total: 521, max: 672
2016-01-14 15:48:32,402 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E2E2D7DC0F3041E7ABC17AC0FA602344
2016-01-14 15:48:32,403 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:32,403 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:32,403 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:32,404 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:32,404 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:32,404 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D262ACA9B2A94E9E8430EB2120B6B3A9
2016-01-14 15:48:32,405 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:32,406 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:32,406 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:32,406 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:32,406 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:32 Completed running task attempt: attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,407 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:32,409 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786512372, Process_Duration=30, Process_End_Time=1452786512402, Read_Duration=7, Tuples_Read=124, Tuples_Written=199, Write_Duration=2][cascading.flow.StepCounters Tuples_Written=199]]
2016-01-14 15:48:32,409 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:32,409 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:32,410 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,410 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786498963_0001_00_000003
2016-01-14 15:48:32,410 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, taskAttemptId=attempt_1452786498963_0001_1_02_000000_0, startTime=1452786512316, finishTime=1452786512408, timeTaken=92, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512372, Process_Duration=30, Process_End_Time=1452786512402, Read_Duration=7, Tuples_Read=124, Tuples_Written=199, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=199
2016-01-14 15:48:32,411 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786498963_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:32,411 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:32,411 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786498963_0001_1_02_000000_0
2016-01-14 15:48:32,412 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:32,413 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:TASK_FINISHED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, taskId=task_1452786498963_0001_1_02_000000, startTime=1452786512316, finishTime=1452786512411, timeTaken=95, status=SUCCEEDED, successfulAttemptID=attempt_1452786498963_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512372, Process_Duration=30, Process_End_Time=1452786512402, Read_Duration=7, Tuples_Read=124, Tuples_Written=199, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=199
2016-01-14 15:48:32,413 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786498963_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:32,413 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] : 1
2016-01-14 15:48:32,413 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:32,414 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:32,415 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:VERTEX_FINISHED]: vertexName=A0C94E7A8AEA45F0A8819AD7A58541D3, vertexId=vertex_1452786498963_0001_1_02, initRequestedTime=1452786499174, initedTime=1452786499200, startRequestedTime=1452786499222, startedTime=1452786499222, finishTime=1452786512413, timeTaken=13191, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=9137, FILE_BYTES_WRITTEN=6659, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=220, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=546832384, OUTPUT_RECORDS=199, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512372, Process_Duration=30, Process_End_Time=1452786512402, Read_Duration=7, Tuples_Read=124, Tuples_Written=199, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=199, vertexStats=firstTaskStartTime=1452786512316, firstTasksToStart=[ task_1452786498963_0001_1_02_000000 ], lastTaskFinishTime=1452786512408, lastTasksToFinish=[ task_1452786498963_0001_1_02_000000 ], minTaskDuration=92, maxTaskDuration=92, avgTaskDuration=92.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786498963_0001_1_02_000000 ], longestDurationTasks=[ task_1452786498963_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:32,415 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:32,415 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:32,416 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786498963_0001_00_000003, stoppedTime=1452786512416, exitStatus=0
2016-01-14 15:48:32,416 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:32,416 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 15:48:32,416 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786498963_0001_1
2016-01-14 15:48:32,416 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786498963_0001_1
2016-01-14 15:48:32,416 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B71CA62645984090B1A2804D20C09E5D for vertex: vertex_1452786498963_0001_1_02 [A0C94E7A8AEA45F0A8819AD7A58541D3]
2016-01-14 15:48:32,418 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:32,421 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786498963_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 15:48:32,421 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786498963_0001_00_000003: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:32,421 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786498963_0001_00_000003, Exiting
2016-01-14 15:48:32,421 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786498963_0001_00_000003
2016-01-14 15:48:32,421 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786498963_0001_00_000003
2016-01-14 15:48:32,423 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786498963_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:32,423 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786498963_0001_00_000003 completed successfully
2016-01-14 15:48:32,428 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786498963_0001_1_00 [E2E2D7DC0F3041E7ABC17AC0FA602344]
2016-01-14 15:48:32,428 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786498963_0001_1_01 [D262ACA9B2A94E9E8430EB2120B6B3A9]
2016-01-14 15:48:32,431 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786498963_0001_1, queueSize=0
2016-01-14 15:48:32,432 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:32,433 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786498963_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786498963_0001_1, startTime=1452786499168, finishTime=1452786512428, timeTaken=13260, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=83619, FILE_BYTES_WRITTEN=25103, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=67, REDUCE_INPUT_RECORDS=124, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=440, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=927, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1639448576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=419, OUTPUT_BYTES=9163, OUTPUT_BYTES_WITH_OVERHEAD=9065, OUTPUT_BYTES_PHYSICAL=9073, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=9073, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=9073, SHUFFLE_BYTES_DECOMPRESSED=9065, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=9073, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=12, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358359511547, Process_Duration=61, Process_End_Time=4358359511608, Read_Duration=10, Tuples_Read=344, Tuples_Written=419, Write_Duration=13, cascading.flow.StepCounters, Tuples_Read=220, Tuples_Written=199
2016-01-14 15:48:32,434 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786498963_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:32,434 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786498963_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:32,434 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786498963_0001_00_000003
2016-01-14 15:48:32,435 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786498963_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:32 Completed Dag: dag_1452786498963_0001_1
2016-01-14 15:48:32,437 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786498963_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:32,438 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786498963_0001_1_post
2016-01-14 15:48:32,438 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,438 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:32,440 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ues/cogroupcompareforward, applicationId=application_1452786498963_0001
2016-01-14 15:48:32,443 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:32,443 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:32,443 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:32,444 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:32,444 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:32,444 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:32,444 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:32,445 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:32,445 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:32,445 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:32,446 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:32,446 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:32,447 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:32,452 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8617E91C245E416A8F5224C02007578D/B9E321C417E248B698509B4776921CE8/.tez/application_1452786498963_0001
2016-01-14 15:48:32,452 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache*ip]  completed in: 00:13.510
2016-01-14 15:48:32,501 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/sortedvaluesplatform/testCoGroupComparatorValues/cogroupcompareforward
2016-01-14 15:48:32,523 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:32,523 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:32,554 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-01-14 15:48:32,557 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.012
2016-01-14 15:48:32,558 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 15:48:32,558 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [apache] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 15:48:32,579 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:32,579 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:32,618 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 15:48:32,632 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-01-14 15:48:32,633 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:32,633 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:32,633 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:32,646 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:32,656 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false
2016-01-14 15:48:32,692 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:32,693 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:32,693 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"]
2016-01-14 15:48:32,693 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  parallel execution of steps is enabled: false
2016-01-14 15:48:32,693 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  executing total steps: 1
2016-01-14 15:48:32,693 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  allocating management threads: 1
2016-01-14 15:48:32,694 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ve/false_null_false_false
2016-01-14 15:48:32,694 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:32,704 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:32,709 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:32,709 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:32,709 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:32,719 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001 doesn't exist and is created
2016-01-14 15:48:32,737 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:32,738 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:32,738 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001
2016-01-14 15:48:32,742 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786512709_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:32,750 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:32,751 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:32,752 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:32,752 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:32,753 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001/localmode-log-dir
2016-01-14 15:48:32,753 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001/localmode-log-dir/history.txt.appattempt_1452786512709_0001_000000, maxErrors=10
2016-01-14 15:48:32,753 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:32,753 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786512709_0001_000000, appSubmitTime=1452786512741, launchTime=1452786512741
2016-01-14 15:48:32,759 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:32,760 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:32,761 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:32,761 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:32,766 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39962
2016-01-14 15:48:32,768 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:32,771 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786512709_0001_000000, startTime=1452786512771
2016-01-14 15:48:32,771 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:32,771 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:32,838 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:32,838 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:32,838 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1452786512709_0001, dagName=[C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-01-14 15:48:32,852 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-01-14 15:48:32,854 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786512709_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001/localmode-log-dir/dag_1452786512709_0001_1.dot
2016-01-14 15:48:32,855 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001/localmode-log-dir/dag_1452786512709_0001_1-tez-dag.pb.txt
2016-01-14 15:48:32,880 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786512709_0001_1
2016-01-14 15:48:32,880 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,880 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-01-14 15:48:32 Running Dag: dag_1452786512709_0001_1
2016-01-14 15:48:32,895 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786512709_0001_1, submitTime=1452786512852
2016-01-14 15:48:32,895 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:32,896 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,896 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:32,898 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786512709_0001_1, initTime=1452786512895
2016-01-14 15:48:32,898 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786512709_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:32,898 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:32,898 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_STARTED]: dagID=dag_1452786512709_0001_1, startTime=1452786512898
2016-01-14 15:48:32,898 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:32,899 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786512709_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:32,899 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FF7D39CC29834167BCDC47B8A32E1623 : {0C68B8C0E004462B9AE58835AEF89075={InputName=0C68B8C0E004462B9AE58835AEF89075}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:32,899 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0C68B8C0E004462B9AE58835AEF89075, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:32,899 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,900 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,900 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,900 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1452786512709_0001, dagName=[C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false
2016-01-14 15:48:32,898 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:32,900 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]. Starting root input initializers: 1
2016-01-14 15:48:32,901 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:32,901 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,903 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0C68B8C0E004462B9AE58835AEF89075 on vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,904 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:32,904 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,904 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,905 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,905 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=027E5FADB04D43FF8048A32FD649448C, vertexId=vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:32,905 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=027E5FADB04D43FF8048A32FD649448C, vertexId=vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,908 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:32,908 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0C68B8C0E004462B9AE58835AEF89075 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:32,909 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=027E5FADB04D43FF8048A32FD649448C, vertexName=F60F05D3CB2842539B07783029236D31, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:32,910 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:32,910 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:32,912 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=027E5FADB04D43FF8048A32FD649448C, vertexId=vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:32,913 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:32,913 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F60F05D3CB2842539B07783029236D31, vertexId=vertex_1452786512709_0001_1_01, initRequestedTime=1452786512901, initedTime=1452786512913, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:32,914 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:32,914 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:32,915 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0C68B8C0E004462B9AE58835AEF89075 on vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,915 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:32,915 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0C68B8C0E004462B9AE58835AEF89075=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:32,915 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] parallelism set to 1
2016-01-14 15:48:32,916 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:32,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786512709_0001_1_00, eventCount=1
2016-01-14 15:48:32,916 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, vertexId=vertex_1452786512709_0001_1_00, initRequestedTime=1452786512899, initedTime=1452786512916, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:32,916 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] with distanceFromRoot: 0
2016-01-14 15:48:32,916 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:32,916 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:32,917 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:32,917 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,917 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786512709_0001_1_00, startRequestedTime=1452786512917, startedTime=1452786512917
2016-01-14 15:48:32,917 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:32,917 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:32,918 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:32,918 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_STARTED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, taskId=task_1452786512709_0001_1_00_000000, scheduledTime=1452786512918, launchTime=1452786512918
2016-01-14 15:48:32,918 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:32,918 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786512709_0001_1_00 for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] numStartedSources: 1 numSources: 1
2016-01-14 15:48:32,918 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] with distanceFromRoot: 1
2016-01-14 15:48:32,918 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786512709_0001, with dag id: dag_1452786512709_0001_1
2016-01-14 15:48:32,919 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:32,919 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FF7D39CC29834167BCDC47B8A32E1623 in vertex: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,920 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F60F05D3CB2842539B07783029236D31 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:32,920 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786512709_0001_1_01, startRequestedTime=1452786512919, startedTime=1452786512919
2016-01-14 15:48:32,920 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:32,920 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:32,920 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: FF7D39CC29834167BCDC47B8A32E1623, VertexParallelism: 1, TaskAttemptID:attempt_1452786512709_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C68B8C0E004462B9AE58835AEF89075, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F60F05D3CB2842539B07783029236D31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:32,921 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:32,921 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:32,924 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:32,924 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:32,925 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:32,925 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786512709_0001_00_000001
2016-01-14 15:48:32,926 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:32,928 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786512709_0001_00_000001
2016-01-14 15:48:32,928 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:32,929 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:32,929 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:32,930 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786512709_0001_00_000001, launchTime=1452786512929
2016-01-14 15:48:32,931 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786512709_0001_00_000001
2016-01-14 15:48:32,932 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786512709_0001_1_00_000000_0] to container: [container_1452786512709_0001_00_000001]
2016-01-14 15:48:32,932 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:32,932 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786512709_0001_00_000001 given task: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,932 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786512709_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,932 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786512709_0001_1_00_000000_0] started. Is using containerId: [container_1452786512709_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:32,933 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0, startTime=1452786512932, containerId=container_1452786512709_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786512709_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:32,933 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:32,933 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:32,933 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:32 Starting to run new task attempt: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,934 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,934 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:32,935 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:32,935 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:32,935 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:32,935 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:32,936 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:32,936 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: FF7D39CC29834167BCDC47B8A32E1623, VertexParallelism: 1, TaskAttemptID:attempt_1452786512709_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C68B8C0E004462B9AE58835AEF89075, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F60F05D3CB2842539B07783029236D31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:32,936 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:32,937 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,937 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:32,937 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,938 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0C68B8C0E004462B9AE58835AEF89075, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:32,938 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,938 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F60F05D3CB2842539B07783029236D31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:32,939 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:32,939 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,939 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:32,940 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,940 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,941 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:32,942 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786512709_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:32,942 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:32,943 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,944 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:32,946 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:32,946 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,946 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,950 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF7D39CC29834167BCDC47B8A32E1623, ordinal: 0
2016-01-14 15:48:32,950 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF7D39CC29834167BCDC47B8A32E1623, mem on start (mb), free: 126, total: 521, max: 672
2016-01-14 15:48:32,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:32,950 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:32,951 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:32,951 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:32,951 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:32,951 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:32,952 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:32,952 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:32,952 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:32,952 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:32,953 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0C68B8C0E004462B9AE58835AEF89075, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:32,953 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F60F05D3CB2842539B07783029236D31, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:32,953 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:32,954 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0C68B8C0E004462B9AE58835AEF89075 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:32,955 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:32,955 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:32,954 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,955 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:32,955 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,956 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:32,956 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,956 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:32,957 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2406cff2
2016-01-14 15:48:32,958 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2406cff2
2016-01-14 15:48:32,958 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:32,958 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:32,959 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:32,961 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 0C68B8C0E004462B9AE58835AEF89075
2016-01-14 15:48:32,961 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: AF08A6C1A8C44B36BEFE05E64CC58DCE
2016-01-14 15:48:32,962 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] AF08A6C1A8C44B36BEFE05E64CC58DCE
2016-01-14 15:48:32,963 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@263bff7b; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2d01f7d9; comparator=cascading.tuple.hadoop.util.TupleComparator@5c36dea4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:32,963 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:32,963 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:32,963 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:32,983 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:32,984 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:32,984 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:32,984 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:32,984 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:32,985 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:32,985 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:32,986 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF7D39CC29834167BCDC47B8A32E1623, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:32,991 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:32,991 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF7D39CC29834167BCDC47B8A32E1623, mem on close (mb), free: 187, total: 516, max: 672
2016-01-14 15:48:32,991 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:32,992 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:32,992 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:32,992 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:32,996 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:32 Completed running task attempt: attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:33,000 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:33,001 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786512709_0001_1_00, eventCount=1
2016-01-14 15:48:33,001 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:33,001 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, taskAttemptId=attempt_1452786512709_0001_1_00_000000_0, startTime=1452786512932, finishTime=1452786513001, timeTaken=69, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512986, Process_Duration=5, Process_End_Time=1452786512991, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:33,001 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:33,002 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786512709_0001_1_00_000000_0
2016-01-14 15:48:33,002 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:33,003 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:33,003 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_FINISHED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, taskId=task_1452786512709_0001_1_00_000000, startTime=1452786512932, finishTime=1452786513002, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1452786512709_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512986, Process_Duration=5, Process_End_Time=1452786512991, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:33,003 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:33,003 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] : 1
2016-01-14 15:48:33,003 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:33,003 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786512986, Process_Duration=5, Process_End_Time=1452786512991, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:33,004 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:33,004 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,004 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786512709_0001_00_000001
2016-01-14 15:48:33,006 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:33,006 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_FINISHED]: vertexName=FF7D39CC29834167BCDC47B8A32E1623, vertexId=vertex_1452786512709_0001_1_00, initRequestedTime=1452786512899, initedTime=1452786512916, startRequestedTime=1452786512917, startedTime=1452786512917, finishTime=1452786513003, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786512986, Process_Duration=5, Process_End_Time=1452786512991, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786512932, firstTasksToStart=[ task_1452786512709_0001_1_00_000000 ], lastTaskFinishTime=1452786513001, lastTasksToFinish=[ task_1452786512709_0001_1_00_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786512709_0001_1_00_000000 ], longestDurationTasks=[ task_1452786512709_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:33,006 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:33,007 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:33,007 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786512709_0001_00_000001, stoppedTime=1452786513007, exitStatus=0
2016-01-14 15:48:33,007 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:33,007 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:33,008 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] attempt: attempt_1452786512709_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:33,008 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F60F05D3CB2842539B07783029236D31
2016-01-14 15:48:33,009 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:33,009 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:33,009 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786512709_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:33,010 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_STARTED]: vertexName=F60F05D3CB2842539B07783029236D31, taskId=task_1452786512709_0001_1_01_000000, scheduledTime=1452786513010, launchTime=1452786513010
2016-01-14 15:48:33,010 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:33,010 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:33,010 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:33,011 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:33,011 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: F60F05D3CB2842539B07783029236D31, VertexParallelism: 1, TaskAttemptID:attempt_1452786512709_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF7D39CC29834167BCDC47B8A32E1623, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=027E5FADB04D43FF8048A32FD649448C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:33,011 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:33,012 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:33,012 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786512709_0001_00_000002
2016-01-14 15:48:33,013 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:33,014 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786512709_0001_00_000002
2016-01-14 15:48:33,014 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:33,015 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:33,015 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:33,016 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786512709_0001_00_000002, launchTime=1452786513015
2016-01-14 15:48:33,015 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786512709_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:33,016 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786512709_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:33,016 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786512709_0001_00_000002
2016-01-14 15:48:33,017 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786512709_0001_00_000001, Exiting
2016-01-14 15:48:33,017 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786512709_0001_00_000001
2016-01-14 15:48:33,017 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786512709_0001_00_000001
2016-01-14 15:48:33,017 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786512709_0001_1_01_000000_0] to container: [container_1452786512709_0001_00_000002]
2016-01-14 15:48:33,020 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:33,020 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786512709_0001_00_000002 given task: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,021 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786512709_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,021 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786512709_0001_1_01_000000_0] started. Is using containerId: [container_1452786512709_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:33,021 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:33 Starting to run new task attempt: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,021 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,021 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,021 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:33,022 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:33,022 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:33,022 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:33,022 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:33,022 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2] comprehensivesort/(1/1) ...ve/false_null_false_false, VertexName: F60F05D3CB2842539B07783029236D31, VertexParallelism: 1, TaskAttemptID:attempt_1452786512709_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF7D39CC29834167BCDC47B8A32E1623, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=027E5FADB04D43FF8048A32FD649448C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:33,022 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:33,021 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786512709_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:33,028 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786512709_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:33,026 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,028 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:33,029 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,021 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F60F05D3CB2842539B07783029236D31, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0, startTime=1452786513020, containerId=container_1452786512709_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786512709_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:33,029 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,028 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786512709_0001_00_000001 completed successfully
2016-01-14 15:48:33,029 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:33,030 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:33,030 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:33,030 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=027E5FADB04D43FF8048A32FD649448C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:33,030 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:33,031 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 027E5FADB04D43FF8048A32FD649448C
2016-01-14 15:48:33,031 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:33,031 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 027E5FADB04D43FF8048A32FD649448C
2016-01-14 15:48:33,031 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:33,031 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FF7D39CC29834167BCDC47B8A32E1623, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:33,035 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:33,035 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,037 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:33,037 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:33,038 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,039 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,041 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F60F05D3CB2842539B07783029236D31, ordinal: 1
2016-01-14 15:48:33,041 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F60F05D3CB2842539B07783029236D31, mem on start (mb), free: 179, total: 516, max: 672
2016-01-14 15:48:33,042 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,042 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:33,042 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:33,044 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:33,045 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 027E5FADB04D43FF8048A32FD649448C
2016-01-14 15:48:33,045 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:33,045 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:33,045 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:33,045 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:33,045 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:33,046 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:33,046 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:33,046 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 027E5FADB04D43FF8048A32FD649448C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:33,046 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FF7D39CC29834167BCDC47B8A32E1623, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:33,046 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:33,047 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,047 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:33,048 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:33,048 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FF7D39CC29834167BCDC47B8A32E1623 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:33,048 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FF7D39CC29834167BCDC47B8A32E1623 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:33,048 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:33,049 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:33,049 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:33,050 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,050 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:33,050 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:33,050 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:33,050 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,056 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:33,056 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786512709_0001_1_00_000000_0_10022, runDuration: 58793, ]
2016-01-14 15:48:33,056 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: AF08A6C1A8C44B36BEFE05E64CC58DCE
2016-01-14 15:48:33,056 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"], id: 027E5FADB04D43FF8048A32FD649448C
2016-01-14 15:48:33,057 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_false_false"]
2016-01-14 15:48:33,057 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] AF08A6C1A8C44B36BEFE05E64CC58DCE, for 1 inputs
2016-01-14 15:48:33,058 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786512709_0001_1_00_000000_0_10022] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786513058, TimeTaken=2, Rate=0.28 MB/s
2016-01-14 15:48:33,058 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,058 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:33,058 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FF7D39CC29834167BCDC47B8A32E1623] #5 in 2ms
2016-01-14 15:48:33,059 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FF7D39CC29834167BCDC47B8A32E1623] #5, status:false, isInterrupted:false
2016-01-14 15:48:33,060 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:33,060 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:33,060 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:33,060 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:33,061 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:33,061 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,061 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F60F05D3CB2842539B07783029236D31, all 1 inputs ready in: 00:00:00.004
2016-01-14 15:48:33,062 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:33,062 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@56a6699e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@47603083
2016-01-14 15:48:33,069 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:33,069 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:33,069 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,070 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786512709_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:33,070 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,070 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F60F05D3CB2842539B07783029236D31, mem on close (mb), free: 175, total: 516, max: 672
2016-01-14 15:48:33,071 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FF7D39CC29834167BCDC47B8A32E1623
2016-01-14 15:48:33,071 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:33,071 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:33,071 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:33,072 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:33,072 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:33 Completed running task attempt: attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,073 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:33,074 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F60F05D3CB2842539B07783029236D31, taskAttemptId=attempt_1452786512709_0001_1_01_000000_0, startTime=1452786513020, finishTime=1452786513073, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513062, Process_Duration=8, Process_End_Time=1452786513070, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:33,074 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786512709_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:33,074 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786512709_0001_1_01_000000_0
2016-01-14 15:48:33,075 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:TASK_FINISHED]: vertexName=F60F05D3CB2842539B07783029236D31, taskId=task_1452786512709_0001_1_01_000000, startTime=1452786513020, finishTime=1452786513074, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452786512709_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513062, Process_Duration=8, Process_End_Time=1452786513070, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:33,075 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:33,075 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786512709_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:33,075 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] : 1
2016-01-14 15:48:33,075 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:33,075 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:33,076 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:33,076 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:VERTEX_FINISHED]: vertexName=F60F05D3CB2842539B07783029236D31, vertexId=vertex_1452786512709_0001_1_01, initRequestedTime=1452786512901, initedTime=1452786512913, startRequestedTime=1452786512919, startedTime=1452786512919, finishTime=1452786513075, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513062, Process_Duration=8, Process_End_Time=1452786513070, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786513020, firstTasksToStart=[ task_1452786512709_0001_1_01_000000 ], lastTaskFinishTime=1452786513073, lastTasksToFinish=[ task_1452786512709_0001_1_01_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786512709_0001_1_01_000000 ], longestDurationTasks=[ task_1452786512709_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:33,077 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:33,077 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:33,077 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:33,077 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786512709_0001_00_000002, stoppedTime=1452786513077, exitStatus=0
2016-01-14 15:48:33,077 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:33,078 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:33,078 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786512709_0001_1
2016-01-14 15:48:33,078 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786512709_0001_1
2016-01-14 15:48:33,078 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786512709_0001_1_00 [FF7D39CC29834167BCDC47B8A32E1623]
2016-01-14 15:48:33,078 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 027E5FADB04D43FF8048A32FD649448C for vertex: vertex_1452786512709_0001_1_01 [F60F05D3CB2842539B07783029236D31]
2016-01-14 15:48:33,078 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:33,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=541589504, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786513062, Process_Duration=8, Process_End_Time=1452786513070, Read_Duration=3, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:33,080 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:33,080 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,081 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786512709_0001_00_000002
2016-01-14 15:48:33,081 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786512709_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:33,081 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786512709_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:33,082 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786512709_0001_00_000002, Exiting
2016-01-14 15:48:33,082 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786512709_0001_00_000002
2016-01-14 15:48:33,082 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786512709_0001_00_000002
2016-01-14 15:48:33,082 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786512709_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:33,082 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786512709_0001_00_000002 completed successfully
2016-01-14 15:48:33,090 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786512709_0001_1, queueSize=0
2016-01-14 15:48:33,090 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:33,091 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786512709_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786512709_0001_1, startTime=1452786512898, finishTime=1452786513088, timeTaken=190, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=859, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1083179008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573026048, Process_Duration=13, Process_End_Time=2905573026061, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:33,091 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786512709_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:33,091 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786512709_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:33,092 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786512709_0001_00_000002
2016-01-14 15:48:33,092 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786512709_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:33 Completed Dag: dag_1452786512709_0001_1
2016-01-14 15:48:33,095 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786512709_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:33,095 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786512709_0001_1_post
2016-01-14 15:48:33,096 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,097 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:33,097 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ve/false_null_false_false, applicationId=application_1452786512709_0001
2016-01-14 15:48:33,100 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:33,101 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:33,101 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:33,101 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:33,101 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:33,101 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:33,102 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:33,103 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:33,103 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:33,103 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:33,104 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:33,104 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:33,104 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:33,110 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C1E60F1339944CA88824B87FECF94B7C/B29E4D21782240C590D3C8AAF37F96C2/.tez/application_1452786512709_0001
2016-01-14 15:48:33,111 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  completed in: 00:00.418
2016-01-14 15:48:33,165 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:33,166 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:33,204 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:48:33,214 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-01-14 15:48:33,215 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:33,215 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:33,215 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:33,230 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:33,239 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false
2016-01-14 15:48:33,278 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:33,278 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:33,278 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"]
2016-01-14 15:48:33,278 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  parallel execution of steps is enabled: false
2016-01-14 15:48:33,278 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  executing total steps: 1
2016-01-14 15:48:33,279 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  allocating management threads: 1
2016-01-14 15:48:33,280 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/true_null_false_false
2016-01-14 15:48:33,280 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:33,289 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:33,294 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:33,294 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:33,294 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:33,304 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001 doesn't exist and is created
2016-01-14 15:48:33,321 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:33,321 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:33,321 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001
2016-01-14 15:48:33,325 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786513294_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:33,333 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:33,334 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:33,335 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:33,335 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:33,335 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001/localmode-log-dir
2016-01-14 15:48:33,336 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001/localmode-log-dir/history.txt.appattempt_1452786513294_0001_000000, maxErrors=10
2016-01-14 15:48:33,336 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:33,336 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786513294_0001_000000, appSubmitTime=1452786513324, launchTime=1452786513325
2016-01-14 15:48:33,341 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:33,344 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:33,346 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:42363
2016-01-14 15:48:33,347 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:33,350 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:33,350 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:33,353 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786513294_0001_000000, startTime=1452786513353
2016-01-14 15:48:33,353 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:33,353 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:33,421 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:33,421 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:33,421 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1452786513294_0001, dagName=[B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-01-14 15:48:33,435 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-01-14 15:48:33,436 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786513294_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001/localmode-log-dir/dag_1452786513294_0001_1.dot
2016-01-14 15:48:33,438 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001/localmode-log-dir/dag_1452786513294_0001_1-tez-dag.pb.txt
2016-01-14 15:48:33,461 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786513294_0001_1
2016-01-14 15:48:33,462 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,462 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-01-14 15:48:33 Running Dag: dag_1452786513294_0001_1
2016-01-14 15:48:33,475 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786513294_0001_1, submitTime=1452786513435
2016-01-14 15:48:33,475 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:33,476 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,477 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:33,478 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786513294_0001_1, initTime=1452786513475
2016-01-14 15:48:33,478 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:33,478 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786513294_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:33,479 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_STARTED]: dagID=dag_1452786513294_0001_1, startTime=1452786513479
2016-01-14 15:48:33,479 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:33,480 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786513294_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:33,480 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F9FC9A7F42574E3F942ED332EBFA8AB2 : {849A9EB1F5744007A9A5F57814780DE9={InputName=849A9EB1F5744007A9A5F57814780DE9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:33,480 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 849A9EB1F5744007A9A5F57814780DE9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:33,480 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,480 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,480 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,479 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:33,481 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]. Starting root input initializers: 1
2016-01-14 15:48:33,482 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1452786513294_0001, dagName=[B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false
2016-01-14 15:48:33,483 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:33,483 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 849A9EB1F5744007A9A5F57814780DE9 on vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,483 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,486 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:33,486 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,487 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,487 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,487 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8199A0D1EEF1492B8D41644EB11897FF, vertexId=vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:33,487 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8199A0D1EEF1492B8D41644EB11897FF, vertexId=vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,486 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:33,488 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 849A9EB1F5744007A9A5F57814780DE9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:33,489 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:33,489 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:33,490 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8199A0D1EEF1492B8D41644EB11897FF, vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:33,490 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8199A0D1EEF1492B8D41644EB11897FF, vertexId=vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:33,492 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:33,492 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:33,492 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, vertexId=vertex_1452786513294_0001_1_01, initRequestedTime=1452786513483, initedTime=1452786513492, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:33,493 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:33,493 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 849A9EB1F5744007A9A5F57814780DE9 on vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,493 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:33,494 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {849A9EB1F5744007A9A5F57814780DE9=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:33,494 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] parallelism set to 1
2016-01-14 15:48:33,494 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:33,494 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786513294_0001, with dag id: dag_1452786513294_0001_1
2016-01-14 15:48:33,495 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786513294_0001_1_00, eventCount=1
2016-01-14 15:48:33,498 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:33,498 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, vertexId=vertex_1452786513294_0001_1_00, initRequestedTime=1452786513480, initedTime=1452786513498, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:33,498 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] with distanceFromRoot: 0
2016-01-14 15:48:33,498 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:33,499 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:33,499 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,500 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:33,500 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786513294_0001_1_00, startRequestedTime=1452786513499, startedTime=1452786513499
2016-01-14 15:48:33,501 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:33,502 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:33,502 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_STARTED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, taskId=task_1452786513294_0001_1_00_000000, scheduledTime=1452786513501, launchTime=1452786513501
2016-01-14 15:48:33,502 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:33,502 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786513294_0001_1_00 for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] numStartedSources: 1 numSources: 1
2016-01-14 15:48:33,502 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] with distanceFromRoot: 1
2016-01-14 15:48:33,503 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:33,503 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F9FC9A7F42574E3F942ED332EBFA8AB2 in vertex: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,503 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DDD4DA62B63D4F7EB1BA28AAD2B36F10 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:33,504 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:33,504 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786513294_0001_1_01, startRequestedTime=1452786513503, startedTime=1452786513503
2016-01-14 15:48:33,504 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:33,505 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: F9FC9A7F42574E3F942ED332EBFA8AB2, VertexParallelism: 1, TaskAttemptID:attempt_1452786513294_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=849A9EB1F5744007A9A5F57814780DE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:33,505 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:33,505 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:33,506 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:33,506 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:33,507 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:33,508 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786513294_0001_00_000001
2016-01-14 15:48:33,508 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:33,510 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786513294_0001_00_000001
2016-01-14 15:48:33,510 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:33,510 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:33,511 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:33,511 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786513294_0001_00_000001, launchTime=1452786513510
2016-01-14 15:48:33,511 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786513294_0001_00_000001
2016-01-14 15:48:33,512 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786513294_0001_1_00_000000_0] to container: [container_1452786513294_0001_00_000001]
2016-01-14 15:48:33,512 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:33,513 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786513294_0001_00_000001 given task: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,513 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786513294_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,513 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786513294_0001_1_00_000000_0] started. Is using containerId: [container_1452786513294_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:33 Starting to run new task attempt: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,514 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:33,514 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,514 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:33,515 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:33,515 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0, startTime=1452786513513, containerId=container_1452786513294_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786513294_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:33,515 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:33,515 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:33,516 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:33,516 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:33,517 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:33,517 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: F9FC9A7F42574E3F942ED332EBFA8AB2, VertexParallelism: 1, TaskAttemptID:attempt_1452786513294_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=849A9EB1F5744007A9A5F57814780DE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:33,517 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:33,515 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:33,518 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,518 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:33,519 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=849A9EB1F5744007A9A5F57814780DE9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:33,520 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,525 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786513294_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:33,525 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:33,526 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,526 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,527 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:33,528 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:33,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:33,528 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,533 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:33,533 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,533 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,534 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:33,535 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,536 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:33,541 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9FC9A7F42574E3F942ED332EBFA8AB2, ordinal: 0
2016-01-14 15:48:33,541 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FC9A7F42574E3F942ED332EBFA8AB2, mem on start (mb), free: 85, total: 516, max: 672
2016-01-14 15:48:33,541 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:33,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:33,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:33,542 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:33,542 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:33,543 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:33,543 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:33,543 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:33,543 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:33,544 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:33,544 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 849A9EB1F5744007A9A5F57814780DE9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:33,544 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DDD4DA62B63D4F7EB1BA28AAD2B36F10, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:33,545 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:33,545 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 849A9EB1F5744007A9A5F57814780DE9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:33,545 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,546 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:33,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:33,546 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:33,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:33,547 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,547 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:33,548 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@66e2cf6e
2016-01-14 15:48:33,549 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@66e2cf6e
2016-01-14 15:48:33,549 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:33,549 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:33,551 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:33,552 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 849A9EB1F5744007A9A5F57814780DE9
2016-01-14 15:48:33,552 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 2C49410EB8674D45AF855319EC703DD2
2016-01-14 15:48:33,553 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 2C49410EB8674D45AF855319EC703DD2
2016-01-14 15:48:33,555 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@30bf57bb; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6218cefd; comparator=cascading.tuple.hadoop.util.TupleComparator@4abd765b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:33,555 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:33,555 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:33,556 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:33,584 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:33,585 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:33,585 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:33,585 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:33,586 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:33,586 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:33,587 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:33,587 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9FC9A7F42574E3F942ED332EBFA8AB2, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:33,592 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:33,592 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9FC9A7F42574E3F942ED332EBFA8AB2, mem on close (mb), free: 190, total: 519, max: 672
2016-01-14 15:48:33,593 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:33,593 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:33,593 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:33,593 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:35,220 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:35 Completed running task attempt: attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:35,224 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:35,224 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786513294_0001_1_00, eventCount=1
2016-01-14 15:48:35,224 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1141, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786513588, Process_Duration=4, Process_End_Time=1452786513592, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:35,225 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, taskAttemptId=attempt_1452786513294_0001_1_00_000000_0, startTime=1452786513513, finishTime=1452786515225, timeTaken=1712, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1141, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513588, Process_Duration=4, Process_End_Time=1452786513592, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:35,225 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:35,225 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:35,226 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,226 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786513294_0001_00_000001
2016-01-14 15:48:35,225 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:35,227 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:35,227 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786513294_0001_1_00_000000_0
2016-01-14 15:48:35,228 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:35,229 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_FINISHED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, taskId=task_1452786513294_0001_1_00_000000, startTime=1452786513513, finishTime=1452786515228, timeTaken=1715, status=SUCCEEDED, successfulAttemptID=attempt_1452786513294_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1141, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513588, Process_Duration=4, Process_End_Time=1452786513592, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:35,229 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:35,229 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] : 1
2016-01-14 15:48:35,229 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:35,230 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:35,230 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_FINISHED]: vertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, vertexId=vertex_1452786513294_0001_1_00, initRequestedTime=1452786513480, initedTime=1452786513498, startRequestedTime=1452786513499, startedTime=1452786513499, finishTime=1452786515229, timeTaken=1730, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=1141, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786513588, Process_Duration=4, Process_End_Time=1452786513592, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786513513, firstTasksToStart=[ task_1452786513294_0001_1_00_000000 ], lastTaskFinishTime=1452786515225, lastTasksToFinish=[ task_1452786513294_0001_1_00_000000 ], minTaskDuration=1712, maxTaskDuration=1712, avgTaskDuration=1712.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786513294_0001_1_00_000000 ], longestDurationTasks=[ task_1452786513294_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:35,230 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:35,231 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:35,231 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786513294_0001_00_000001, stoppedTime=1452786515231, exitStatus=0
2016-01-14 15:48:35,231 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:35,231 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:35,232 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] attempt: attempt_1452786513294_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:35,232 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DDD4DA62B63D4F7EB1BA28AAD2B36F10
2016-01-14 15:48:35,232 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:35,233 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:35,233 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786513294_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:35,233 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_STARTED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, taskId=task_1452786513294_0001_1_01_000000, scheduledTime=1452786515233, launchTime=1452786515233
2016-01-14 15:48:35,233 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:35,234 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:35,234 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:35,234 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: DDD4DA62B63D4F7EB1BA28AAD2B36F10, VertexParallelism: 1, TaskAttemptID:attempt_1452786513294_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8199A0D1EEF1492B8D41644EB11897FF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:35,235 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:35,235 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:35,233 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:35,236 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786513294_0001_00_000002
2016-01-14 15:48:35,236 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:35,238 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786513294_0001_00_000002
2016-01-14 15:48:35,238 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:35,239 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786513294_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:35,239 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786513294_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:35,239 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:35,239 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786513294_0001_00_000001, Exiting
2016-01-14 15:48:35,239 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786513294_0001_00_000001
2016-01-14 15:48:35,240 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786513294_0001_00_000001
2016-01-14 15:48:35,241 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:35,240 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786513294_0001_00_000002, launchTime=1452786515239
2016-01-14 15:48:35,241 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786513294_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:35,242 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786513294_0001_00_000001 completed successfully
2016-01-14 15:48:35,242 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:35,242 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786513294_0001_00_000002
2016-01-14 15:48:35,243 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786513294_0001_1_01_000000_0] to container: [container_1452786513294_0001_00_000002]
2016-01-14 15:48:35,243 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:35,243 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786513294_0001_00_000002 given task: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,243 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786513294_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,244 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786513294_0001_1_01_000000_0] started. Is using containerId: [container_1452786513294_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:35 Starting to run new task attempt: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,244 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,244 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,244 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:35,244 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:35,244 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:35,245 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:35,245 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:35,244 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:35,245 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08] comprehensivesort/(1/1) ...ive/true_null_false_false, VertexName: DDD4DA62B63D4F7EB1BA28AAD2B36F10, VertexParallelism: 1, TaskAttemptID:attempt_1452786513294_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8199A0D1EEF1492B8D41644EB11897FF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:35,245 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:35,245 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0, startTime=1452786515243, containerId=container_1452786513294_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786513294_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:35,246 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:35,246 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:35,247 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,248 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:35,248 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,248 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786513294_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:35,249 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F9FC9A7F42574E3F942ED332EBFA8AB2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:35,249 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8199A0D1EEF1492B8D41644EB11897FF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:35,249 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,250 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:35,250 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,250 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:35,251 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8199A0D1EEF1492B8D41644EB11897FF
2016-01-14 15:48:35,251 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:35,251 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8199A0D1EEF1492B8D41644EB11897FF
2016-01-14 15:48:35,253 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:35,253 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:35,253 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,253 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:35,255 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,264 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DDD4DA62B63D4F7EB1BA28AAD2B36F10, ordinal: 1
2016-01-14 15:48:35,264 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:35,264 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8199A0D1EEF1492B8D41644EB11897FF
2016-01-14 15:48:35,264 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DDD4DA62B63D4F7EB1BA28AAD2B36F10, mem on start (mb), free: 87, total: 428, max: 672
2016-01-14 15:48:35,265 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,265 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:35,265 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:35,265 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:35,266 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:35,266 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:35,266 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:35,266 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:35,267 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:35,267 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:35,267 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8199A0D1EEF1492B8D41644EB11897FF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:35,268 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F9FC9A7F42574E3F942ED332EBFA8AB2, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:35,268 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:35,268 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,269 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:35,269 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:35,269 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F9FC9A7F42574E3F942ED332EBFA8AB2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:35,269 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F9FC9A7F42574E3F942ED332EBFA8AB2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:35,269 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:35,270 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:35,270 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:35,271 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,272 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:35,273 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:35,273 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:35,273 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,274 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786513294_0001_1_00_000000_0_10029, runDuration: 1693322, ]
2016-01-14 15:48:35,275 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786513294_0001_1_00_000000_0_10029] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786515275, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:35,275 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,275 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:35,275 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F9FC9A7F42574E3F942ED332EBFA8AB2] #6 in 1ms
2016-01-14 15:48:35,276 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F9FC9A7F42574E3F942ED332EBFA8AB2] #6, status:false, isInterrupted:false
2016-01-14 15:48:35,277 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:35,278 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:35,278 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:35,278 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:35,278 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:35,279 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 2C49410EB8674D45AF855319EC703DD2
2016-01-14 15:48:35,279 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"], id: 8199A0D1EEF1492B8D41644EB11897FF
2016-01-14 15:48:35,279 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:35,279 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,279 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_false_false"]
2016-01-14 15:48:35,280 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:35,280 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 2C49410EB8674D45AF855319EC703DD2, for 1 inputs
2016-01-14 15:48:35,280 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DDD4DA62B63D4F7EB1BA28AAD2B36F10, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:35,281 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@563e5f74; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@18ece4e8
2016-01-14 15:48:35,285 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:35,286 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:35,286 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,287 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786513294_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:35,287 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,288 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DDD4DA62B63D4F7EB1BA28AAD2B36F10, mem on close (mb), free: 84, total: 428, max: 672
2016-01-14 15:48:35,288 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F9FC9A7F42574E3F942ED332EBFA8AB2
2016-01-14 15:48:35,288 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:35,289 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:35,289 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:35,289 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:35,290 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:35 Completed running task attempt: attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,291 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:35,292 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, taskAttemptId=attempt_1452786513294_0001_1_01_000000_0, startTime=1452786515243, finishTime=1452786515291, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515281, Process_Duration=6, Process_End_Time=1452786515287, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:35,292 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:35,292 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786513294_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:35,292 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786513294_0001_1_01_000000_0
2016-01-14 15:48:35,292 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:35,293 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:35,293 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:TASK_FINISHED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, taskId=task_1452786513294_0001_1_01_000000, startTime=1452786515243, finishTime=1452786515292, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452786513294_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515281, Process_Duration=6, Process_End_Time=1452786515287, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:35,294 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786513294_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:35,294 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] : 1
2016-01-14 15:48:35,294 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:35,294 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786515281, Process_Duration=6, Process_End_Time=1452786515287, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:35,295 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:35,296 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,297 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:35,297 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:VERTEX_FINISHED]: vertexName=DDD4DA62B63D4F7EB1BA28AAD2B36F10, vertexId=vertex_1452786513294_0001_1_01, initRequestedTime=1452786513483, initedTime=1452786513492, startRequestedTime=1452786513503, startedTime=1452786513503, finishTime=1452786515294, timeTaken=1791, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=448790528, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515281, Process_Duration=6, Process_End_Time=1452786515287, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786515243, firstTasksToStart=[ task_1452786513294_0001_1_01_000000 ], lastTaskFinishTime=1452786515291, lastTasksToFinish=[ task_1452786513294_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786513294_0001_1_01_000000 ], longestDurationTasks=[ task_1452786513294_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:35,297 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:35,297 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786513294_0001_00_000002
2016-01-14 15:48:35,298 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:35,298 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786513294_0001_00_000002, stoppedTime=1452786515298, exitStatus=0
2016-01-14 15:48:35,298 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:35,298 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:35,299 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:35,299 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786513294_0001_1
2016-01-14 15:48:35,299 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786513294_0001_1
2016-01-14 15:48:35,300 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786513294_0001_1_00 [F9FC9A7F42574E3F942ED332EBFA8AB2]
2016-01-14 15:48:35,300 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8199A0D1EEF1492B8D41644EB11897FF for vertex: vertex_1452786513294_0001_1_01 [DDD4DA62B63D4F7EB1BA28AAD2B36F10]
2016-01-14 15:48:35,308 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786513294_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:35,308 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786513294_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:35,309 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786513294_0001_00_000002, Exiting
2016-01-14 15:48:35,309 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786513294_0001_00_000002
2016-01-14 15:48:35,309 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786513294_0001_00_000002
2016-01-14 15:48:35,310 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786513294_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:35,310 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786513294_0001_00_000002 completed successfully
2016-01-14 15:48:35,312 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786513294_0001_1, queueSize=0
2016-01-14 15:48:35,312 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:35,313 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786513294_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786513294_0001_1, startTime=1452786513479, finishTime=1452786515309, timeTaken=1830, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=1895, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=352, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=897581056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573028869, Process_Duration=10, Process_End_Time=2905573028879, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:35,313 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786513294_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:35,313 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786513294_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:35,314 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786513294_0001_00_000002
2016-01-14 15:48:35,314 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786513294_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:35 Completed Dag: dag_1452786513294_0001_1
2016-01-14 15:48:35,315 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786513294_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:35,315 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786513294_0001_1_post
2016-01-14 15:48:35,315 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,316 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:35,321 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_null_false_false, applicationId=application_1452786513294_0001
2016-01-14 15:48:35,323 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:35,324 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:35,324 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:35,324 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:35,324 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:35,324 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:35,325 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:35,325 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:35,325 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:35,326 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:35,326 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:35,326 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:35,327 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:35,332 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B1672382CC2C45A0950D5B9CF710701E/ACD70EB1DDCF4AE68B0F07EEDACCDF08/.tez/application_1452786513294_0001
2016-01-14 15:48:35,333 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  completed in: 00:02.055
2016-01-14 15:48:35,394 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:35,395 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:35,438 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 15:48:35,447 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.034
2016-01-14 15:48:35,448 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:35,448 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:35,448 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:35,460 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:35,469 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false
2016-01-14 15:48:35,508 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:35,508 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:35,508 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"]
2016-01-14 15:48:35,509 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  parallel execution of steps is enabled: false
2016-01-14 15:48:35,509 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  executing total steps: 1
2016-01-14 15:48:35,509 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  allocating management threads: 1
2016-01-14 15:48:35,511 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_true_false
2016-01-14 15:48:35,511 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:35,519 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:35,524 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:35,524 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:35,524 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:35,533 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001 doesn't exist and is created
2016-01-14 15:48:35,548 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:35,548 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:35,549 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001
2016-01-14 15:48:35,552 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786515524_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:35,560 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:35,561 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:35,562 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:35,562 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:35,563 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001/localmode-log-dir
2016-01-14 15:48:35,563 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001/localmode-log-dir/history.txt.appattempt_1452786515524_0001_000000, maxErrors=10
2016-01-14 15:48:35,563 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:35,563 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786515524_0001_000000, appSubmitTime=1452786515552, launchTime=1452786515552
2016-01-14 15:48:35,566 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:35,567 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:35,569 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:35,569 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:35,571 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55639
2016-01-14 15:48:35,575 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:35,577 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786515524_0001_000000, startTime=1452786515577
2016-01-14 15:48:35,577 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:35,577 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:35,649 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:35,649 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:35,649 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1452786515524_0001, dagName=[AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-01-14 15:48:35,661 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-01-14 15:48:35,663 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786515524_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001/localmode-log-dir/dag_1452786515524_0001_1.dot
2016-01-14 15:48:35,663 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001/localmode-log-dir/dag_1452786515524_0001_1-tez-dag.pb.txt
2016-01-14 15:48:35,687 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786515524_0001_1
2016-01-14 15:48:35,687 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,687 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-01-14 15:48:35 Running Dag: dag_1452786515524_0001_1
2016-01-14 15:48:35,700 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786515524_0001_1, submitTime=1452786515661
2016-01-14 15:48:35,700 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:35,701 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,701 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:35,702 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:35,701 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786515524_0001_1, initTime=1452786515700
2016-01-14 15:48:35,702 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786515524_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:35,703 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_STARTED]: dagID=dag_1452786515524_0001_1, startTime=1452786515702
2016-01-14 15:48:35,703 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:35,703 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:35,703 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786515524_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:35,703 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8A68EE5B98D4C73B4655A05B3EDFAC4 : {A60FAE0D99484F9EA9A870B3E7488EAA={InputName=A60FAE0D99484F9EA9A870B3E7488EAA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:35,703 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A60FAE0D99484F9EA9A870B3E7488EAA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:35,703 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1452786515524_0001, dagName=[AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false
2016-01-14 15:48:35,704 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,704 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,704 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,704 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]. Starting root input initializers: 1
2016-01-14 15:48:35,705 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:35,705 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,708 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:35,708 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,709 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,709 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,709 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EBE87D8F458D4319A7C2A0AFCDA08D1F, vertexId=vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:35,709 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EBE87D8F458D4319A7C2A0AFCDA08D1F, vertexId=vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,711 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EBE87D8F458D4319A7C2A0AFCDA08D1F, vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:35,711 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EBE87D8F458D4319A7C2A0AFCDA08D1F, vertexId=vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,712 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A60FAE0D99484F9EA9A870B3E7488EAA on vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,713 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:35,713 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, vertexId=vertex_1452786515524_0001_1_01, initRequestedTime=1452786515705, initedTime=1452786515712, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:35,713 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:35,715 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:35,716 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A60FAE0D99484F9EA9A870B3E7488EAA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:35,716 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:35,717 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786515524_0001, with dag id: dag_1452786515524_0001_1
2016-01-14 15:48:35,717 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:35,722 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:35,722 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A60FAE0D99484F9EA9A870B3E7488EAA on vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,722 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:35,723 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A60FAE0D99484F9EA9A870B3E7488EAA=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:35,723 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] parallelism set to 1
2016-01-14 15:48:35,723 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:35,723 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786515524_0001_1_00, eventCount=1
2016-01-14 15:48:35,724 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, vertexId=vertex_1452786515524_0001_1_00, initRequestedTime=1452786515703, initedTime=1452786515724, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:35,724 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:35,724 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] with distanceFromRoot: 0
2016-01-14 15:48:35,725 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:35,725 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,725 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,726 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786515524_0001_1_00, startRequestedTime=1452786515725, startedTime=1452786515725
2016-01-14 15:48:35,726 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:35,726 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:35,727 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:35,727 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_STARTED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, taskId=task_1452786515524_0001_1_00_000000, scheduledTime=1452786515726, launchTime=1452786515726
2016-01-14 15:48:35,727 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:35,727 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786515524_0001_1_00 for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] numStartedSources: 1 numSources: 1
2016-01-14 15:48:35,728 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] with distanceFromRoot: 1
2016-01-14 15:48:35,728 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,728 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C8A68EE5B98D4C73B4655A05B3EDFAC4 in vertex: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,728 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AC8EAB16353346EF94CD2B3A8A13BF1F with 1 source tasks and 1 pending tasks
2016-01-14 15:48:35,729 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786515524_0001_1_01, startRequestedTime=1452786515728, startedTime=1452786515728
2016-01-14 15:48:35,729 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:35,729 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:35,730 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: C8A68EE5B98D4C73B4655A05B3EDFAC4, VertexParallelism: 1, TaskAttemptID:attempt_1452786515524_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A60FAE0D99484F9EA9A870B3E7488EAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:35,730 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:35,730 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:35,731 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:35,731 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:35,731 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:35,732 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786515524_0001_00_000001
2016-01-14 15:48:35,732 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:35,733 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786515524_0001_00_000001
2016-01-14 15:48:35,733 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:35,734 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:35,734 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:35,734 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786515524_0001_00_000001, launchTime=1452786515734
2016-01-14 15:48:35,735 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786515524_0001_00_000001
2016-01-14 15:48:35,737 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786515524_0001_1_00_000000_0] to container: [container_1452786515524_0001_00_000001]
2016-01-14 15:48:35,737 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:35,737 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786515524_0001_1_00_000000_0] started. Is using containerId: [container_1452786515524_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:35,737 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0, startTime=1452786515737, containerId=container_1452786515524_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786515524_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:35,738 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:35,737 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:35,738 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786515524_0001_00_000001 given task: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,738 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:35,738 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786515524_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35 Starting to run new task attempt: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,739 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,739 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,739 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:35,739 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:35,740 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:35,740 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:35,740 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:35,740 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: C8A68EE5B98D4C73B4655A05B3EDFAC4, VertexParallelism: 1, TaskAttemptID:attempt_1452786515524_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A60FAE0D99484F9EA9A870B3E7488EAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:35,741 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:35,741 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,741 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:35,741 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,742 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A60FAE0D99484F9EA9A870B3E7488EAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:35,742 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:35,742 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,742 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,744 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,744 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:35,745 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786515524_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:35,745 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:35,745 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,746 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,745 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:35,746 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,747 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:35,748 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:35,748 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,749 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,756 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8A68EE5B98D4C73B4655A05B3EDFAC4, ordinal: 0
2016-01-14 15:48:35,756 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8A68EE5B98D4C73B4655A05B3EDFAC4, mem on start (mb), free: 217, total: 430, max: 672
2016-01-14 15:48:35,757 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,757 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:35,757 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:35,757 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:35,757 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:35,758 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:35,758 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:35,758 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:35,758 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:35,759 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:35,759 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A60FAE0D99484F9EA9A870B3E7488EAA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:35,759 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AC8EAB16353346EF94CD2B3A8A13BF1F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:35,759 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:35,759 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A60FAE0D99484F9EA9A870B3E7488EAA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:35,760 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:35,760 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:35,759 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,760 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:35,761 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,761 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:35,761 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,761 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:35,762 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2fc0f146
2016-01-14 15:48:35,763 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2fc0f146
2016-01-14 15:48:35,763 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:35,763 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:35,764 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:35,764 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: A60FAE0D99484F9EA9A870B3E7488EAA
2016-01-14 15:48:35,764 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: A6E9E074BD1D414ABDACF2055C924228
2016-01-14 15:48:35,765 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A6E9E074BD1D414ABDACF2055C924228
2016-01-14 15:48:35,765 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@15ad5b59; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@28483b2d; comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@1e251c04; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:35,766 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:35,766 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:35,766 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:35,780 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:35,780 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:35,780 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:35,780 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:35,781 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:35,781 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:35,781 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:35,782 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8A68EE5B98D4C73B4655A05B3EDFAC4, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:35,788 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,788 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8A68EE5B98D4C73B4655A05B3EDFAC4, mem on close (mb), free: 115, total: 430, max: 672
2016-01-14 15:48:35,789 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:35,789 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:35,789 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:35,789 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:35,792 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:35 Completed running task attempt: attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,797 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:35,798 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786515782, Process_Duration=6, Process_End_Time=1452786515788, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:35,798 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786515524_0001_1_00, eventCount=1
2016-01-14 15:48:35,799 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:35,799 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,799 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:35,799 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, taskAttemptId=attempt_1452786515524_0001_1_00_000000_0, startTime=1452786515737, finishTime=1452786515799, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515782, Process_Duration=6, Process_End_Time=1452786515788, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:35,799 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:35,800 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786515524_0001_00_000001
2016-01-14 15:48:35,800 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786515524_0001_1_00_000000_0
2016-01-14 15:48:35,800 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:35,801 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:35,801 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_FINISHED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, taskId=task_1452786515524_0001_1_00_000000, startTime=1452786515737, finishTime=1452786515800, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1452786515524_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515782, Process_Duration=6, Process_End_Time=1452786515788, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:35,801 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:35,801 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] : 1
2016-01-14 15:48:35,801 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,803 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:35,805 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_FINISHED]: vertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, vertexId=vertex_1452786515524_0001_1_00, initRequestedTime=1452786515703, initedTime=1452786515724, startRequestedTime=1452786515725, startedTime=1452786515725, finishTime=1452786515801, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515782, Process_Duration=6, Process_End_Time=1452786515788, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786515737, firstTasksToStart=[ task_1452786515524_0001_1_00_000000 ], lastTaskFinishTime=1452786515799, lastTasksToFinish=[ task_1452786515524_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786515524_0001_1_00_000000 ], longestDurationTasks=[ task_1452786515524_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:35,805 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:35,805 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:35,805 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786515524_0001_00_000001, stoppedTime=1452786515805, exitStatus=0
2016-01-14 15:48:35,805 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:35,805 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] attempt: attempt_1452786515524_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:35,806 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AC8EAB16353346EF94CD2B3A8A13BF1F
2016-01-14 15:48:35,806 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,806 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:35,806 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786515524_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:35,806 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:35,807 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:35,808 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_STARTED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, taskId=task_1452786515524_0001_1_01_000000, scheduledTime=1452786515806, launchTime=1452786515806
2016-01-14 15:48:35,808 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:35,808 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,809 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:35,809 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: AC8EAB16353346EF94CD2B3A8A13BF1F, VertexParallelism: 1, TaskAttemptID:attempt_1452786515524_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EBE87D8F458D4319A7C2A0AFCDA08D1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:35,810 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:35,810 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:35,811 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786515524_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:35,811 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786515524_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:35,811 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786515524_0001_00_000001, Exiting
2016-01-14 15:48:35,811 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786515524_0001_00_000001
2016-01-14 15:48:35,811 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786515524_0001_00_000001
2016-01-14 15:48:35,812 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786515524_0001_00_000002
2016-01-14 15:48:35,812 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:35,813 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786515524_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:35,814 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786515524_0001_00_000001 completed successfully
2016-01-14 15:48:35,814 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:35,814 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786515524_0001_00_000002
2016-01-14 15:48:35,815 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:35,818 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:35,818 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:35,818 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786515524_0001_00_000002, launchTime=1452786515818
2016-01-14 15:48:35,819 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786515524_0001_00_000002
2016-01-14 15:48:35,820 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786515524_0001_1_01_000000_0] to container: [container_1452786515524_0001_00_000002]
2016-01-14 15:48:35,820 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:35,821 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786515524_0001_1_01_000000_0] started. Is using containerId: [container_1452786515524_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:35,821 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0, startTime=1452786515820, containerId=container_1452786515524_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786515524_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:35,821 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:35,821 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:35,821 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:35,821 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786515524_0001_00_000002 given task: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,822 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786515524_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35 Starting to run new task attempt: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,822 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,822 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,822 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:35,822 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:35,823 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:35,823 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:35,823 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:35,823 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403] comprehensivesort/(1/1) ...ive/false_null_true_false, VertexName: AC8EAB16353346EF94CD2B3A8A13BF1F, VertexParallelism: 1, TaskAttemptID:attempt_1452786515524_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EBE87D8F458D4319A7C2A0AFCDA08D1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:35,823 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:35,824 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,824 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:35,824 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,825 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C8A68EE5B98D4C73B4655A05B3EDFAC4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:35,825 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,828 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786515524_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:35,828 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EBE87D8F458D4319A7C2A0AFCDA08D1F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:35,829 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:35,829 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EBE87D8F458D4319A7C2A0AFCDA08D1F
2016-01-14 15:48:35,830 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:35,830 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EBE87D8F458D4319A7C2A0AFCDA08D1F
2016-01-14 15:48:35,827 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:35,831 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,832 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:35,834 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:35,834 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:35,835 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,835 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,840 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AC8EAB16353346EF94CD2B3A8A13BF1F, ordinal: 1
2016-01-14 15:48:35,840 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC8EAB16353346EF94CD2B3A8A13BF1F, mem on start (mb), free: 107, total: 430, max: 672
2016-01-14 15:48:35,840 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:35,840 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:35,841 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:35,843 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:35,843 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EBE87D8F458D4319A7C2A0AFCDA08D1F
2016-01-14 15:48:35,843 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:35,844 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:35,844 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:35,844 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:35,844 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:35,844 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:35,845 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:35,845 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EBE87D8F458D4319A7C2A0AFCDA08D1F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:35,845 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C8A68EE5B98D4C73B4655A05B3EDFAC4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:35,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:35,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C8A68EE5B98D4C73B4655A05B3EDFAC4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:35,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,845 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:35,847 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:35,848 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:35,848 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:35,849 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C8A68EE5B98D4C73B4655A05B3EDFAC4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:35,849 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:35,849 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:35,850 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:35,851 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,851 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:35,851 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,851 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786515524_0001_1_00_000000_0_10035, runDuration: 48607, ]
2016-01-14 15:48:35,853 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786515524_0001_1_00_000000_0_10035] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786515853, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:35,853 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,854 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:35,855 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C8A68EE5B98D4C73B4655A05B3EDFAC4] #7 in 3ms
2016-01-14 15:48:35,856 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:35,857 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: A6E9E074BD1D414ABDACF2055C924228
2016-01-14 15:48:35,857 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"], id: EBE87D8F458D4319A7C2A0AFCDA08D1F
2016-01-14 15:48:35,857 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_null_true_false"]
2016-01-14 15:48:35,857 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] A6E9E074BD1D414ABDACF2055C924228, for 1 inputs
2016-01-14 15:48:35,857 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C8A68EE5B98D4C73B4655A05B3EDFAC4] #7, status:false, isInterrupted:false
2016-01-14 15:48:35,859 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:35,859 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:35,859 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:35,860 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:35,860 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:35,860 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,861 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AC8EAB16353346EF94CD2B3A8A13BF1F, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:48:35,861 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:35,861 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3ae0242f; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5167a7db
2016-01-14 15:48:35,865 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:35,866 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:35,866 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,866 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786515524_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:35,867 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,867 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC8EAB16353346EF94CD2B3A8A13BF1F, mem on close (mb), free: 103, total: 430, max: 672
2016-01-14 15:48:35,867 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C8A68EE5B98D4C73B4655A05B3EDFAC4
2016-01-14 15:48:35,867 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:35,868 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:35,868 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:35,868 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:35,868 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:35 Completed running task attempt: attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,869 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:35,870 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:35,870 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, taskAttemptId=attempt_1452786515524_0001_1_01_000000_0, startTime=1452786515820, finishTime=1452786515869, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515861, Process_Duration=6, Process_End_Time=1452786515867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:35,870 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786515524_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:35,870 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786515524_0001_1_01_000000_0
2016-01-14 15:48:35,870 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:35,871 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:35,871 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:TASK_FINISHED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, taskId=task_1452786515524_0001_1_01_000000, startTime=1452786515820, finishTime=1452786515870, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1452786515524_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515861, Process_Duration=6, Process_End_Time=1452786515867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:35,871 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786515524_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:35,871 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] : 1
2016-01-14 15:48:35,871 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,872 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:35,873 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:VERTEX_FINISHED]: vertexName=AC8EAB16353346EF94CD2B3A8A13BF1F, vertexId=vertex_1452786515524_0001_1_01, initRequestedTime=1452786515705, initedTime=1452786515712, startRequestedTime=1452786515728, startedTime=1452786515728, finishTime=1452786515871, timeTaken=143, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786515861, Process_Duration=6, Process_End_Time=1452786515867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786515820, firstTasksToStart=[ task_1452786515524_0001_1_01_000000 ], lastTaskFinishTime=1452786515869, lastTasksToFinish=[ task_1452786515524_0001_1_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786515524_0001_1_01_000000 ], longestDurationTasks=[ task_1452786515524_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:35,873 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:35,873 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:35,873 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786515524_0001_00_000002, stoppedTime=1452786515873, exitStatus=0
2016-01-14 15:48:35,873 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:35,874 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:35,874 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:35,874 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786515524_0001_1
2016-01-14 15:48:35,874 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786515524_0001_1
2016-01-14 15:48:35,874 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786515524_0001_1_00 [C8A68EE5B98D4C73B4655A05B3EDFAC4]
2016-01-14 15:48:35,874 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EBE87D8F458D4319A7C2A0AFCDA08D1F for vertex: vertex_1452786515524_0001_1_01 [AC8EAB16353346EF94CD2B3A8A13BF1F]
2016-01-14 15:48:35,875 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=450887680, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786515861, Process_Duration=6, Process_End_Time=1452786515867, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:35,875 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:35,875 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,876 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786515524_0001_00_000002
2016-01-14 15:48:35,876 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786515524_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:35,876 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786515524_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:35,877 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786515524_0001_00_000002, Exiting
2016-01-14 15:48:35,877 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786515524_0001_00_000002
2016-01-14 15:48:35,877 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786515524_0001_00_000002
2016-01-14 15:48:35,878 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786515524_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:35,878 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786515524_0001_00_000002 completed successfully
2016-01-14 15:48:35,886 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786515524_0001_1, queueSize=0
2016-01-14 15:48:35,886 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:35,887 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786515524_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786515524_0001_1, startTime=1452786515702, finishTime=1452786515884, timeTaken=182, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=859, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=901775360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573031643, Process_Duration=12, Process_End_Time=2905573031655, Read_Duration=4, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:35,887 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786515524_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:35,887 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786515524_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:35,887 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786515524_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:35,887 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786515524_0001_00_000002
2016-01-14 15:48:35 Completed Dag: dag_1452786515524_0001_1
2016-01-14 15:48:35,887 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786515524_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:35,887 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786515524_0001_1_post
2016-01-14 15:48:35,887 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:35,888 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:35,891 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/false_null_true_false, applicationId=application_1452786515524_0001
2016-01-14 15:48:35,894 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:35,895 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:35,895 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:35,895 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:35,895 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:35,895 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:35,895 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:35,896 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:35,896 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:35,896 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:35,897 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:35,897 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:35,897 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:35,902 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AFE05690B7AE4162B6C8FD1FAE9649B7/BCF37711D2EA49D6B4A036BCEAD80403/.tez/application_1452786515524_0001
2016-01-14 15:48:35,903 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  completed in: 00:00.394
2016-01-14 15:48:35,957 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:35,957 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:35,993 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 15:48:36,005 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-01-14 15:48:36,006 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:36,006 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:36,006 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:36,026 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:36,037 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false
2016-01-14 15:48:36,076 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:36,076 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:36,076 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"]
2016-01-14 15:48:36,076 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  parallel execution of steps is enabled: false
2016-01-14 15:48:36,077 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  executing total steps: 1
2016-01-14 15:48:36,077 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  allocating management threads: 1
2016-01-14 15:48:36,078 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_true_false
2016-01-14 15:48:36,078 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:36,087 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:36,091 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:36,091 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:36,091 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:36,100 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001 doesn't exist and is created
2016-01-14 15:48:36,120 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:36,121 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:36,121 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001
2016-01-14 15:48:36,124 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786516091_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:36,137 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:36,138 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:36,140 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:36,140 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:36,140 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001/localmode-log-dir
2016-01-14 15:48:36,141 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001/localmode-log-dir/history.txt.appattempt_1452786516091_0001_000000, maxErrors=10
2016-01-14 15:48:36,141 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:36,141 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786516091_0001_000000, appSubmitTime=1452786516124, launchTime=1452786516124
2016-01-14 15:48:36,145 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:36,147 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:36,148 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:36,148 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:36,153 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:48679
2016-01-14 15:48:36,155 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:36,157 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786516091_0001_000000, startTime=1452786516157
2016-01-14 15:48:36,157 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:36,157 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:36,221 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:36,221 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:36,221 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1452786516091_0001, dagName=[73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-01-14 15:48:36,235 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-01-14 15:48:36,236 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786516091_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001/localmode-log-dir/dag_1452786516091_0001_1.dot
2016-01-14 15:48:36,237 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001/localmode-log-dir/dag_1452786516091_0001_1-tez-dag.pb.txt
2016-01-14 15:48:36,259 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786516091_0001_1
2016-01-14 15:48:36,260 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:36,260 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-01-14 15:48:36 Running Dag: dag_1452786516091_0001_1
2016-01-14 15:48:36,277 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:36,278 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786516091_0001_1, submitTime=1452786516235
2016-01-14 15:48:36,279 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,279 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:36,279 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:36,281 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786516091_0001_1, initTime=1452786516278
2016-01-14 15:48:36,281 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786516091_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:36,282 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_STARTED]: dagID=dag_1452786516091_0001_1, startTime=1452786516282
2016-01-14 15:48:36,282 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:36,282 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786516091_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:36,282 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:36,283 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BCA7656A60EE4599B0476B1E8894585D : {EF76503B14824744B92323B77FD4D5A5={InputName=EF76503B14824744B92323B77FD4D5A5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:36,283 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EF76503B14824744B92323B77FD4D5A5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:36,283 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,283 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,283 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,284 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]. Starting root input initializers: 1
2016-01-14 15:48:36,284 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1452786516091_0001, dagName=[73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false
2016-01-14 15:48:36,285 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:36,285 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,287 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EF76503B14824744B92323B77FD4D5A5 on vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,288 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:36,289 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,289 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:36,290 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,290 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,290 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0CE4D6ED85304C68A60E06871115D775, vertexId=vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:36,291 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0CE4D6ED85304C68A60E06871115D775, vertexId=vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,292 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EF76503B14824744B92323B77FD4D5A5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:36,293 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:36,293 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:36,294 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0CE4D6ED85304C68A60E06871115D775, vertexName=B14CEFDAAB9446D9B508118DD3D287B2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:36,294 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0CE4D6ED85304C68A60E06871115D775, vertexId=vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:36,296 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:36,296 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, vertexId=vertex_1452786516091_0001_1_01, initRequestedTime=1452786516285, initedTime=1452786516296, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:36,297 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:36,297 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:36,297 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EF76503B14824744B92323B77FD4D5A5 on vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,298 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:36,298 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EF76503B14824744B92323B77FD4D5A5=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:36,299 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] parallelism set to 1
2016-01-14 15:48:36,299 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:36,299 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786516091_0001_1_00, eventCount=1
2016-01-14 15:48:36,299 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786516091_0001, with dag id: dag_1452786516091_0001_1
2016-01-14 15:48:36,302 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:36,302 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, vertexId=vertex_1452786516091_0001_1_00, initRequestedTime=1452786516283, initedTime=1452786516302, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:36,304 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] with distanceFromRoot: 0
2016-01-14 15:48:36,304 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:36,304 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:36,305 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,305 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786516091_0001_1_00, startRequestedTime=1452786516304, startedTime=1452786516304
2016-01-14 15:48:36,305 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:36,305 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:36,306 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_STARTED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, taskId=task_1452786516091_0001_1_00_000000, scheduledTime=1452786516306, launchTime=1452786516306
2016-01-14 15:48:36,306 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:36,306 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:36,309 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786516091_0001_1_00 for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] numStartedSources: 1 numSources: 1
2016-01-14 15:48:36,309 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] with distanceFromRoot: 1
2016-01-14 15:48:36,310 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:36,310 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BCA7656A60EE4599B0476B1E8894585D in vertex: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,311 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B14CEFDAAB9446D9B508118DD3D287B2 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:36,311 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:36,311 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786516091_0001_1_01, startRequestedTime=1452786516310, startedTime=1452786516310
2016-01-14 15:48:36,311 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:36,312 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: BCA7656A60EE4599B0476B1E8894585D, VertexParallelism: 1, TaskAttemptID:attempt_1452786516091_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF76503B14824744B92323B77FD4D5A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B14CEFDAAB9446D9B508118DD3D287B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:36,313 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:36,313 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:36,313 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:36,314 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:36,314 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:36,315 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786516091_0001_00_000001
2016-01-14 15:48:36,315 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:36,317 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786516091_0001_00_000001
2016-01-14 15:48:36,317 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:36,317 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:36,317 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786516091_0001_00_000001, launchTime=1452786516317
2016-01-14 15:48:36,317 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:36,319 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786516091_0001_00_000001
2016-01-14 15:48:36,320 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786516091_0001_1_00_000000_0] to container: [container_1452786516091_0001_00_000001]
2016-01-14 15:48:36,320 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:36,321 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786516091_0001_00_000001 given task: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,321 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786516091_0001_1_00_000000_0] started. Is using containerId: [container_1452786516091_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:36,321 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0, startTime=1452786516321, containerId=container_1452786516091_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786516091_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:36,321 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:36,321 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:36,321 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786516091_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36 Starting to run new task attempt: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,321 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:36,322 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,322 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:36,322 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:36,323 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:36,323 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:36,323 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:36,323 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:36,323 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: BCA7656A60EE4599B0476B1E8894585D, VertexParallelism: 1, TaskAttemptID:attempt_1452786516091_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF76503B14824744B92323B77FD4D5A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B14CEFDAAB9446D9B508118DD3D287B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:36,323 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:36,324 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,324 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:36,324 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:36,325 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EF76503B14824744B92323B77FD4D5A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:36,325 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:36,329 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:36,330 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B14CEFDAAB9446D9B508118DD3D287B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:36,331 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:36,331 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,332 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:36,332 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,333 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:36,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:36,335 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,335 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,326 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786516091_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:36,341 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:36,341 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,341 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,344 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BCA7656A60EE4599B0476B1E8894585D, ordinal: 0
2016-01-14 15:48:36,344 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCA7656A60EE4599B0476B1E8894585D, mem on start (mb), free: 232, total: 420, max: 672
2016-01-14 15:48:36,344 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:36,344 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:36,344 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:36,344 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:36,345 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:36,345 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:36,345 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:36,346 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:36,346 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:36,347 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:36,347 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B14CEFDAAB9446D9B508118DD3D287B2, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:36,347 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EF76503B14824744B92323B77FD4D5A5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:36,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:36,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,348 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:36,349 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,348 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EF76503B14824744B92323B77FD4D5A5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:36,349 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:36,350 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:36,350 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:36,351 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,351 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:36,352 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1216bde8
2016-01-14 15:48:36,353 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1216bde8
2016-01-14 15:48:36,353 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:36,353 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:36,356 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:36,356 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: EF76503B14824744B92323B77FD4D5A5
2016-01-14 15:48:36,356 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: DA36AE3B34AF49A88E76AF2FE760DC0A
2016-01-14 15:48:36,357 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] DA36AE3B34AF49A88E76AF2FE760DC0A
2016-01-14 15:48:36,358 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5d571114; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7f2267eb; comparator=cascading.tuple.hadoop.util.ReverseTupleComparator@2bd4b682; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:36,359 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 15:48:36,359 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 15:48:36,360 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:36,373 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:36,374 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:36,374 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:36,374 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:36,374 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:36,375 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:36,375 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:36,375 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BCA7656A60EE4599B0476B1E8894585D, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:36,382 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:36,382 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BCA7656A60EE4599B0476B1E8894585D, mem on close (mb), free: 130, total: 420, max: 672
2016-01-14 15:48:36,382 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:36,382 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:36,382 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2016-01-14 15:48:36,382 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:37,444 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:37,445 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:38,028 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:38 Completed running task attempt: attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:38,034 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:38,035 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786516376, Process_Duration=6, Process_End_Time=1452786516382, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:38,035 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:38,036 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,035 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786516091_0001_1_00, eventCount=1
2016-01-14 15:48:38,036 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786516091_0001_00_000001
2016-01-14 15:48:38,037 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:38,037 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, taskAttemptId=attempt_1452786516091_0001_1_00_000000_0, startTime=1452786516321, finishTime=1452786518036, timeTaken=1715, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786516376, Process_Duration=6, Process_End_Time=1452786516382, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:38,038 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:38,038 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786516091_0001_1_00_000000_0
2016-01-14 15:48:38,038 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:38,039 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:38,039 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_FINISHED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, taskId=task_1452786516091_0001_1_00_000000, startTime=1452786516321, finishTime=1452786518038, timeTaken=1717, status=SUCCEEDED, successfulAttemptID=attempt_1452786516091_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786516376, Process_Duration=6, Process_End_Time=1452786516382, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:38,039 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:38,039 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] : 1
2016-01-14 15:48:38,039 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:38,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_FINISHED]: vertexName=BCA7656A60EE4599B0476B1E8894585D, vertexId=vertex_1452786516091_0001_1_00, initRequestedTime=1452786516283, initedTime=1452786516302, startRequestedTime=1452786516304, startedTime=1452786516304, finishTime=1452786518039, timeTaken=1735, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=625, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786516376, Process_Duration=6, Process_End_Time=1452786516382, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786516321, firstTasksToStart=[ task_1452786516091_0001_1_00_000000 ], lastTaskFinishTime=1452786518036, lastTasksToFinish=[ task_1452786516091_0001_1_00_000000 ], minTaskDuration=1715, maxTaskDuration=1715, avgTaskDuration=1715.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786516091_0001_1_00_000000 ], longestDurationTasks=[ task_1452786516091_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:38,041 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:38,041 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:38,041 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786516091_0001_00_000001, stoppedTime=1452786518041, exitStatus=0
2016-01-14 15:48:38,041 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:38,041 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] attempt: attempt_1452786516091_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:38,041 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B14CEFDAAB9446D9B508118DD3D287B2
2016-01-14 15:48:38,042 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:38,042 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:38,042 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:38,043 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:38,044 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786516091_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:38,044 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_STARTED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, taskId=task_1452786516091_0001_1_01_000000, scheduledTime=1452786518044, launchTime=1452786518044
2016-01-14 15:48:38,044 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:38,045 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:38,045 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:38,046 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: B14CEFDAAB9446D9B508118DD3D287B2, VertexParallelism: 1, TaskAttemptID:attempt_1452786516091_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BCA7656A60EE4599B0476B1E8894585D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0CE4D6ED85304C68A60E06871115D775, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:38,044 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:38,046 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:38,047 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:38,047 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786516091_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:38,047 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786516091_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:38,047 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786516091_0001_00_000001, Exiting
2016-01-14 15:48:38,047 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786516091_0001_00_000001
2016-01-14 15:48:38,048 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786516091_0001_00_000001
2016-01-14 15:48:38,048 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786516091_0001_00_000002
2016-01-14 15:48:38,050 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:38,051 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786516091_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:38,051 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786516091_0001_00_000002
2016-01-14 15:48:38,051 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786516091_0001_00_000001 completed successfully
2016-01-14 15:48:38,052 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:38,052 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:38,054 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:38,054 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786516091_0001_00_000002, launchTime=1452786518054
2016-01-14 15:48:38,054 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:38,055 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786516091_0001_00_000002
2016-01-14 15:48:38,056 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786516091_0001_1_01_000000_0] to container: [container_1452786516091_0001_00_000002]
2016-01-14 15:48:38,056 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:38,056 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786516091_0001_00_000002 given task: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,056 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786516091_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,056 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786516091_0001_1_01_000000_0] started. Is using containerId: [container_1452786516091_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:38 Starting to run new task attempt: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,056 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,057 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:38,057 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0, startTime=1452786518056, containerId=container_1452786516091_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786516091_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:38,057 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:38,057 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:38,058 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,059 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:38,059 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:38,059 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:38,059 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:38,059 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:38,060 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1] comprehensivesort/(1/1) ...sive/true_null_true_false, VertexName: B14CEFDAAB9446D9B508118DD3D287B2, VertexParallelism: 1, TaskAttemptID:attempt_1452786516091_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BCA7656A60EE4599B0476B1E8894585D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0CE4D6ED85304C68A60E06871115D775, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:38,060 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:38,061 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,061 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:38,061 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,062 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786516091_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:38,062 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BCA7656A60EE4599B0476B1E8894585D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:38,062 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0CE4D6ED85304C68A60E06871115D775, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:38,062 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,063 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:38,063 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:38,063 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,064 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0CE4D6ED85304C68A60E06871115D775
2016-01-14 15:48:38,064 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:38,065 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0CE4D6ED85304C68A60E06871115D775
2016-01-14 15:48:38,066 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:38,068 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:38,068 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:38,068 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,070 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,077 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B14CEFDAAB9446D9B508118DD3D287B2, ordinal: 1
2016-01-14 15:48:38,077 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B14CEFDAAB9446D9B508118DD3D287B2, mem on start (mb), free: 254, total: 432, max: 672
2016-01-14 15:48:38,077 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,077 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:38,078 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:38,079 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:38,079 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0CE4D6ED85304C68A60E06871115D775
2016-01-14 15:48:38,079 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:38,079 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:38,080 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:38,080 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:38,080 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:38,080 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:38,080 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:38,080 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0CE4D6ED85304C68A60E06871115D775, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:38,080 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BCA7656A60EE4599B0476B1E8894585D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:38,080 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:38,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BCA7656A60EE4599B0476B1E8894585D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:38,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:38,081 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:38,082 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:38,082 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:38,083 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BCA7656A60EE4599B0476B1E8894585D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:38,083 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:38,083 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:38,083 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:38,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,084 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:38,084 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,086 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786516091_0001_1_00_000000_0_10040, runDuration: 1701074, ]
2016-01-14 15:48:38,087 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:38,088 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: DA36AE3B34AF49A88E76AF2FE760DC0A
2016-01-14 15:48:38,088 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786516091_0001_1_00_000000_0_10040] to DISK_DIRECT, CompressedSize=593, DecompressedSize=589,EndTime=1452786518088, TimeTaken=1, Rate=0.57 MB/s
2016-01-14 15:48:38,088 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"], id: 0CE4D6ED85304C68A60E06871115D775
2016-01-14 15:48:38,088 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,088 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_null_true_false"]
2016-01-14 15:48:38,089 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] DA36AE3B34AF49A88E76AF2FE760DC0A, for 1 inputs
2016-01-14 15:48:38,088 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:38,090 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BCA7656A60EE4599B0476B1E8894585D] #8 in 3ms
2016-01-14 15:48:38,090 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BCA7656A60EE4599B0476B1E8894585D] #8, status:false, isInterrupted:false
2016-01-14 15:48:38,091 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:38,092 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 593 bytes from disk
2016-01-14 15:48:38,092 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:38,092 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:38,092 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 580 bytes
2016-01-14 15:48:38,092 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,092 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:38,094 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B14CEFDAAB9446D9B508118DD3D287B2, all 1 inputs ready in: 00:00:00.004
2016-01-14 15:48:38,096 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@36db6ea8; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3df3bb
2016-01-14 15:48:38,100 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:38,100 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:38,101 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,101 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786516091_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:38,102 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,102 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B14CEFDAAB9446D9B508118DD3D287B2, mem on close (mb), free: 249, total: 432, max: 672
2016-01-14 15:48:38,102 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:38,102 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:38,102 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BCA7656A60EE4599B0476B1E8894585D
2016-01-14 15:48:38,103 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:38,103 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:38,103 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:38,104 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:38,104 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:38 Completed running task attempt: attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,105 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:38,107 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786518094, Process_Duration=8, Process_End_Time=1452786518102, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:38,107 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, taskAttemptId=attempt_1452786516091_0001_1_01_000000_0, startTime=1452786518056, finishTime=1452786518107, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518094, Process_Duration=8, Process_End_Time=1452786518102, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:38,107 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:38,107 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,107 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:38,107 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786516091_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:38,108 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786516091_0001_1_01_000000_0
2016-01-14 15:48:38,108 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:38,108 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:38,109 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:TASK_FINISHED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, taskId=task_1452786516091_0001_1_01_000000, startTime=1452786518056, finishTime=1452786518108, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1452786516091_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518094, Process_Duration=8, Process_End_Time=1452786518102, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:38,109 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786516091_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:38,109 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] : 1
2016-01-14 15:48:38,109 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:38,110 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:38,110 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:VERTEX_FINISHED]: vertexName=B14CEFDAAB9446D9B508118DD3D287B2, vertexId=vertex_1452786516091_0001_1_01, initRequestedTime=1452786516285, initedTime=1452786516296, startRequestedTime=1452786516310, startedTime=1452786516310, finishTime=1452786518109, timeTaken=1799, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=625, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518094, Process_Duration=8, Process_End_Time=1452786518102, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786518056, firstTasksToStart=[ task_1452786516091_0001_1_01_000000 ], lastTaskFinishTime=1452786518107, lastTasksToFinish=[ task_1452786516091_0001_1_01_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786516091_0001_1_01_000000 ], longestDurationTasks=[ task_1452786516091_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:38,110 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:38,111 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:38,111 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786516091_0001_00_000002, stoppedTime=1452786518111, exitStatus=0
2016-01-14 15:48:38,111 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:38,108 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786516091_0001_00_000002
2016-01-14 15:48:38,111 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:38,112 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:38,112 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786516091_0001_1
2016-01-14 15:48:38,112 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786516091_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:38,112 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786516091_0001_1
2016-01-14 15:48:38,112 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0CE4D6ED85304C68A60E06871115D775 for vertex: vertex_1452786516091_0001_1_01 [B14CEFDAAB9446D9B508118DD3D287B2]
2016-01-14 15:48:38,116 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786516091_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:38,116 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786516091_0001_00_000002, Exiting
2016-01-14 15:48:38,116 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786516091_0001_00_000002
2016-01-14 15:48:38,116 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786516091_0001_00_000002
2016-01-14 15:48:38,117 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786516091_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:38,118 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786516091_0001_00_000002 completed successfully
2016-01-14 15:48:38,122 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786516091_0001_1_00 [BCA7656A60EE4599B0476B1E8894585D]
2016-01-14 15:48:38,124 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786516091_0001_1, queueSize=0
2016-01-14 15:48:38,124 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:38,125 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786516091_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786516091_0001_1, startTime=1452786516282, finishTime=1452786518122, timeTaken=1840, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=989, FILE_BYTES_WRITTEN=859, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=389, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=905969664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=629, OUTPUT_BYTES_WITH_OVERHEAD=589, OUTPUT_BYTES_PHYSICAL=593, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=593, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=593, SHUFFLE_BYTES_DECOMPRESSED=589, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=593, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573034470, Process_Duration=14, Process_End_Time=2905573034484, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:38,126 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786516091_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:38,126 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786516091_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:38,127 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786516091_0001_00_000002
2016-01-14 15:48:38,127 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786516091_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:38 Completed Dag: dag_1452786516091_0001_1
2016-01-14 15:48:38,130 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786516091_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:38,130 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786516091_0001_1_post
2016-01-14 15:48:38,130 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,130 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:38,131 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sive/true_null_true_false, applicationId=application_1452786516091_0001
2016-01-14 15:48:38,133 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:38,133 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:38,133 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:38,133 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:38,133 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:38,134 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:38,134 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:38,134 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:38,135 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:38,135 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:38,135 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:38,136 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:38,136 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:38,140 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/73B5B8DE96274323869CAE53B06F92EC/DA830A613B9043D1B9627C3A24B7AEF1/.tez/application_1452786516091_0001
2016-01-14 15:48:38,141 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  completed in: 00:02.065
2016-01-14 15:48:38,194 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:38,194 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:38,240 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 15:48:38,253 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.040
2016-01-14 15:48:38,254 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:38,254 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:38,254 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:38,266 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:38,274 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false
2016-01-14 15:48:38,310 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:38,311 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:38,311 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"]
2016-01-14 15:48:38,312 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  parallel execution of steps is enabled: false
2016-01-14 15:48:38,312 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  executing total steps: 1
2016-01-14 15:48:38,313 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  allocating management threads: 1
2016-01-14 15:48:38,314 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...e/false_false_false_false
2016-01-14 15:48:38,314 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:38,322 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:38,327 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:38,327 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:38,327 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:38,338 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001 doesn't exist and is created
2016-01-14 15:48:38,356 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:38,356 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:38,357 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001
2016-01-14 15:48:38,361 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786518327_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:38,369 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:38,370 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:38,372 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:38,373 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:38,373 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001/localmode-log-dir
2016-01-14 15:48:38,374 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001/localmode-log-dir/history.txt.appattempt_1452786518327_0001_000000, maxErrors=10
2016-01-14 15:48:38,374 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:38,375 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786518327_0001_000000, appSubmitTime=1452786518361, launchTime=1452786518361
2016-01-14 15:48:38,381 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:38,381 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:38,383 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:38,383 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:38,388 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40196
2016-01-14 15:48:38,390 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:38,392 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786518327_0001_000000, startTime=1452786518392
2016-01-14 15:48:38,392 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:38,392 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:38,457 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:38,457 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:38,457 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1452786518327_0001, dagName=[E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false
2016-01-14 15:48:38,470 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false
2016-01-14 15:48:38,471 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786518327_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001/localmode-log-dir/dag_1452786518327_0001_1.dot
2016-01-14 15:48:38,472 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001/localmode-log-dir/dag_1452786518327_0001_1-tez-dag.pb.txt
2016-01-14 15:48:38,490 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786518327_0001_1
2016-01-14 15:48:38,490 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,490 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false
2016-01-14 15:48:38 Running Dag: dag_1452786518327_0001_1
2016-01-14 15:48:38,504 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:38,505 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786518327_0001_1, submitTime=1452786518470
2016-01-14 15:48:38,505 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,506 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:38,506 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:38,506 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786518327_0001_1, initTime=1452786518505
2016-01-14 15:48:38,507 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518327_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:38,508 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_STARTED]: dagID=dag_1452786518327_0001_1, startTime=1452786518507
2016-01-14 15:48:38,508 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:38,508 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518327_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:38,508 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:38,508 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EF1ACD2574CE4C67966F6DB70D92535D : {3FC9FD5A60764766900DB085D1ABA2C1={InputName=3FC9FD5A60764766900DB085D1ABA2C1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:38,509 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3FC9FD5A60764766900DB085D1ABA2C1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:38,509 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1452786518327_0001, dagName=[E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false
2016-01-14 15:48:38,509 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,509 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,509 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,510 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]. Starting root input initializers: 1
2016-01-14 15:48:38,510 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:38,510 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3FC9FD5A60764766900DB085D1ABA2C1 on vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,510 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,514 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:38,514 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3FC9FD5A60764766900DB085D1ABA2C1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:38,514 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:38,515 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,515 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:38,515 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,515 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,516 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9DC90CBA103E4575BC5169E30B287DA7, vertexId=vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:38,516 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9DC90CBA103E4575BC5169E30B287DA7, vertexId=vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,515 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:38,518 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9DC90CBA103E4575BC5169E30B287DA7, vertexName=B720CBD51C0C48DF97E86274C0E700E4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:38,518 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9DC90CBA103E4575BC5169E30B287DA7, vertexId=vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,518 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:38,519 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3FC9FD5A60764766900DB085D1ABA2C1 on vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,520 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:38,520 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, vertexId=vertex_1452786518327_0001_1_01, initRequestedTime=1452786518510, initedTime=1452786518520, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:38,521 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:38,521 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:38,521 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3FC9FD5A60764766900DB085D1ABA2C1=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:38,521 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] parallelism set to 1
2016-01-14 15:48:38,521 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:38,522 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786518327_0001_1_00, eventCount=1
2016-01-14 15:48:38,522 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, vertexId=vertex_1452786518327_0001_1_00, initRequestedTime=1452786518508, initedTime=1452786518522, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:38,522 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:38,523 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] with distanceFromRoot: 0
2016-01-14 15:48:38,523 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:38,523 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,523 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,523 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:38,524 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786518327_0001_1_00, startRequestedTime=1452786518523, startedTime=1452786518523
2016-01-14 15:48:38,524 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:38,524 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:38,525 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_STARTED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, taskId=task_1452786518327_0001_1_00_000000, scheduledTime=1452786518524, launchTime=1452786518524
2016-01-14 15:48:38,525 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:38,525 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786518327_0001_1_00 for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] numStartedSources: 1 numSources: 1
2016-01-14 15:48:38,525 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] with distanceFromRoot: 1
2016-01-14 15:48:38,526 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,526 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EF1ACD2574CE4C67966F6DB70D92535D in vertex: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,526 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B720CBD51C0C48DF97E86274C0E700E4 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:38,526 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:38,527 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786518327_0001_1_01, startRequestedTime=1452786518526, startedTime=1452786518526
2016-01-14 15:48:38,527 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:38,527 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: EF1ACD2574CE4C67966F6DB70D92535D, VertexParallelism: 1, TaskAttemptID:attempt_1452786518327_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3FC9FD5A60764766900DB085D1ABA2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B720CBD51C0C48DF97E86274C0E700E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:38,527 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:38,528 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:38,528 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:38,529 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:38,529 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:38,529 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786518327_0001, with dag id: dag_1452786518327_0001_1
2016-01-14 15:48:38,529 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786518327_0001_00_000001
2016-01-14 15:48:38,530 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:38,531 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786518327_0001_00_000001
2016-01-14 15:48:38,531 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:38,532 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:38,532 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:38,532 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786518327_0001_00_000001, launchTime=1452786518532
2016-01-14 15:48:38,536 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518327_0001_00_000001
2016-01-14 15:48:38,537 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786518327_0001_1_00_000000_0] to container: [container_1452786518327_0001_00_000001]
2016-01-14 15:48:38,537 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:38,537 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786518327_0001_1_00_000000_0] started. Is using containerId: [container_1452786518327_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:38,538 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0, startTime=1452786518537, containerId=container_1452786518327_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786518327_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:38,538 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:38,538 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:38,538 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:38,538 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786518327_0001_00_000001 given task: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,538 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518327_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38 Starting to run new task attempt: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,539 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,539 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,539 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:38,539 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:38,539 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:38,539 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:38,540 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:38,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: EF1ACD2574CE4C67966F6DB70D92535D, VertexParallelism: 1, TaskAttemptID:attempt_1452786518327_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3FC9FD5A60764766900DB085D1ABA2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B720CBD51C0C48DF97E86274C0E700E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:38,540 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:38,540 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,540 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:38,541 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,543 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786518327_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:38,543 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,543 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3FC9FD5A60764766900DB085D1ABA2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:38,543 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B720CBD51C0C48DF97E86274C0E700E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:38,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:38,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:38,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,544 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,544 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,546 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:38,546 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:38,546 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,547 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,549 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:38,549 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,549 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,553 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EF1ACD2574CE4C67966F6DB70D92535D, ordinal: 0
2016-01-14 15:48:38,553 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EF1ACD2574CE4C67966F6DB70D92535D, mem on start (mb), free: 160, total: 432, max: 672
2016-01-14 15:48:38,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:38,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:38,554 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:38,554 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:38,554 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:38,554 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:38,554 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:38,555 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:38,555 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:38,555 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3FC9FD5A60764766900DB085D1ABA2C1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:38,555 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B720CBD51C0C48DF97E86274C0E700E4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:38,555 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:38,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3FC9FD5A60764766900DB085D1ABA2C1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:38,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,556 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:38,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,556 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:38,557 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:38,557 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:38,557 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,557 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:38,558 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4f7ebeec
2016-01-14 15:48:38,559 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4f7ebeec
2016-01-14 15:48:38,559 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:38,560 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:38,561 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:38,562 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 3FC9FD5A60764766900DB085D1ABA2C1
2016-01-14 15:48:38,562 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 2ED7AC534669403FB9B25EDE2400DAD6
2016-01-14 15:48:38,563 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 2ED7AC534669403FB9B25EDE2400DAD6
2016-01-14 15:48:38,564 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@34adeb80; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6403b85; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@63039bf7; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:38,565 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:38,565 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:38,565 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:38,584 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:38,585 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:38,585 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:38,586 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:38,586 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:38,587 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:38,587 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:38,588 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EF1ACD2574CE4C67966F6DB70D92535D, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:38,592 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,592 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EF1ACD2574CE4C67966F6DB70D92535D, mem on close (mb), free: 202, total: 432, max: 672
2016-01-14 15:48:38,593 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:38,593 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:38,593 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:38,594 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:38,597 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:38 Completed running task attempt: attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,601 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:38,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786518327_0001_1_00, eventCount=1
2016-01-14 15:48:38,602 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:38,602 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, taskAttemptId=attempt_1452786518327_0001_1_00_000000_0, startTime=1452786518537, finishTime=1452786518602, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518588, Process_Duration=4, Process_End_Time=1452786518592, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:38,603 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:38,603 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786518327_0001_1_00_000000_0
2016-01-14 15:48:38,603 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:38,603 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_FINISHED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, taskId=task_1452786518327_0001_1_00_000000, startTime=1452786518537, finishTime=1452786518603, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1452786518327_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518588, Process_Duration=4, Process_End_Time=1452786518592, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:38,604 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:38,604 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] : 1
2016-01-14 15:48:38,604 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,605 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:38,606 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_FINISHED]: vertexName=EF1ACD2574CE4C67966F6DB70D92535D, vertexId=vertex_1452786518327_0001_1_00, initRequestedTime=1452786518508, initedTime=1452786518522, startRequestedTime=1452786518523, startedTime=1452786518523, finishTime=1452786518604, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518588, Process_Duration=4, Process_End_Time=1452786518592, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786518537, firstTasksToStart=[ task_1452786518327_0001_1_00_000000 ], lastTaskFinishTime=1452786518602, lastTasksToFinish=[ task_1452786518327_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786518327_0001_1_00_000000 ], longestDurationTasks=[ task_1452786518327_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:38,606 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:38,606 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:38,606 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] attempt: attempt_1452786518327_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:38,606 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B720CBD51C0C48DF97E86274C0E700E4
2016-01-14 15:48:38,606 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,607 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:38,607 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786518327_0001_00_000001, stoppedTime=1452786518607, exitStatus=0
2016-01-14 15:48:38,607 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:38,607 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:38,608 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_STARTED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, taskId=task_1452786518327_0001_1_01_000000, scheduledTime=1452786518607, launchTime=1452786518607
2016-01-14 15:48:38,608 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:38,607 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:38,608 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786518327_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:38,608 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:38,608 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,608 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:38,609 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: B720CBD51C0C48DF97E86274C0E700E4, VertexParallelism: 1, TaskAttemptID:attempt_1452786518327_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF1ACD2574CE4C67966F6DB70D92535D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9DC90CBA103E4575BC5169E30B287DA7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:38,609 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:38,609 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:38,608 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786518588, Process_Duration=4, Process_End_Time=1452786518592, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:38,609 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:38,609 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,612 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786518327_0001_00_000002
2016-01-14 15:48:38,612 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:38,614 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786518327_0001_00_000002
2016-01-14 15:48:38,614 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:38,610 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518327_0001_00_000001
2016-01-14 15:48:38,615 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786518327_0001_00_000002, launchTime=1452786518614
2016-01-14 15:48:38,615 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:38,615 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:38,616 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518327_0001_00_000002
2016-01-14 15:48:38,615 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786518327_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:38,617 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518327_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:38,617 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786518327_0001_1_01_000000_0] to container: [container_1452786518327_0001_00_000002]
2016-01-14 15:48:38,617 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786518327_0001_00_000001, Exiting
2016-01-14 15:48:38,617 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786518327_0001_00_000001
2016-01-14 15:48:38,617 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786518327_0001_00_000001
2016-01-14 15:48:38,617 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:38,619 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786518327_0001_00_000002 given task: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,619 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518327_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,619 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786518327_0001_1_01_000000_0] started. Is using containerId: [container_1452786518327_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:38 Starting to run new task attempt: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,619 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,619 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0, startTime=1452786518619, containerId=container_1452786518327_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786518327_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:38,619 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:38,620 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,620 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:38,620 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:38,620 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:38,620 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:38,621 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:38,621 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5] comprehensivesort/(1/1) ...e/false_false_false_false, VertexName: B720CBD51C0C48DF97E86274C0E700E4, VertexParallelism: 1, TaskAttemptID:attempt_1452786518327_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EF1ACD2574CE4C67966F6DB70D92535D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9DC90CBA103E4575BC5169E30B287DA7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:38,621 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:38,621 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786518327_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:38,622 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786518327_0001_00_000001 completed successfully
2016-01-14 15:48:38,620 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:38,622 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786518327_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:38,622 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,623 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:38,623 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,624 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:38,624 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:38,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EF1ACD2574CE4C67966F6DB70D92535D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:38,625 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9DC90CBA103E4575BC5169E30B287DA7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:38,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:38,626 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9DC90CBA103E4575BC5169E30B287DA7
2016-01-14 15:48:38,626 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:38,627 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9DC90CBA103E4575BC5169E30B287DA7
2016-01-14 15:48:38,627 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:38,628 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:38,628 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,630 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:38,630 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:38,631 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,632 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,636 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B720CBD51C0C48DF97E86274C0E700E4, ordinal: 1
2016-01-14 15:48:38,636 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B720CBD51C0C48DF97E86274C0E700E4, mem on start (mb), free: 195, total: 432, max: 672
2016-01-14 15:48:38,636 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:38,637 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:38,637 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:38,639 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:38,639 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9DC90CBA103E4575BC5169E30B287DA7
2016-01-14 15:48:38,640 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:38,640 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:38,640 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:38,640 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:38,640 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:38,640 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:38,641 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:38,641 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9DC90CBA103E4575BC5169E30B287DA7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:38,641 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EF1ACD2574CE4C67966F6DB70D92535D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:38,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:38,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EF1ACD2574CE4C67966F6DB70D92535D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:38,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,641 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:38,642 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:38,642 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:38,642 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:38,643 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EF1ACD2574CE4C67966F6DB70D92535D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:38,643 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:38,643 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:38,643 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:38,644 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,645 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:38,645 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,646 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786518327_0001_1_00_000000_0_10047, runDuration: 54999, ]
2016-01-14 15:48:38,648 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786518327_0001_1_00_000000_0_10047] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786518648, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:38,648 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:38,648 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,649 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 2ED7AC534669403FB9B25EDE2400DAD6
2016-01-14 15:48:38,649 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"], id: 9DC90CBA103E4575BC5169E30B287DA7
2016-01-14 15:48:38,649 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_false_false_false"]
2016-01-14 15:48:38,649 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 2ED7AC534669403FB9B25EDE2400DAD6, for 1 inputs
2016-01-14 15:48:38,649 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:38,650 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [EF1ACD2574CE4C67966F6DB70D92535D] #9 in 3ms
2016-01-14 15:48:38,651 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EF1ACD2574CE4C67966F6DB70D92535D] #9, status:false, isInterrupted:false
2016-01-14 15:48:38,651 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:38,652 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:38,652 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:38,652 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:38,652 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:38,652 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,652 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:38,652 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B720CBD51C0C48DF97E86274C0E700E4, all 1 inputs ready in: 00:00:00.003
2016-01-14 15:48:38,653 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@22d642b0; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@20b442ae
2016-01-14 15:48:38,658 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:38,659 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:38,659 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,659 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786518327_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:38,660 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,660 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B720CBD51C0C48DF97E86274C0E700E4, mem on close (mb), free: 192, total: 432, max: 672
2016-01-14 15:48:38,660 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EF1ACD2574CE4C67966F6DB70D92535D
2016-01-14 15:48:38,661 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:38,661 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:38,661 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:38,661 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:38,661 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:38 Completed running task attempt: attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,662 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:38,663 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, taskAttemptId=attempt_1452786518327_0001_1_01_000000_0, startTime=1452786518619, finishTime=1452786518663, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518653, Process_Duration=7, Process_End_Time=1452786518660, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:38,663 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518327_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:38,664 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:38,664 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786518327_0001_1_01_000000_0
2016-01-14 15:48:38,664 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:38,665 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:TASK_FINISHED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, taskId=task_1452786518327_0001_1_01_000000, startTime=1452786518619, finishTime=1452786518664, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452786518327_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518653, Process_Duration=7, Process_End_Time=1452786518660, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:38,665 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518327_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:38,665 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] : 1
2016-01-14 15:48:38,665 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:38,665 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,667 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:VERTEX_FINISHED]: vertexName=B720CBD51C0C48DF97E86274C0E700E4, vertexId=vertex_1452786518327_0001_1_01, initRequestedTime=1452786518510, initedTime=1452786518520, startRequestedTime=1452786518526, startedTime=1452786518526, finishTime=1452786518665, timeTaken=139, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786518653, Process_Duration=7, Process_End_Time=1452786518660, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786518619, firstTasksToStart=[ task_1452786518327_0001_1_01_000000 ], lastTaskFinishTime=1452786518663, lastTasksToFinish=[ task_1452786518327_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786518327_0001_1_01_000000 ], longestDurationTasks=[ task_1452786518327_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:38,667 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:38,667 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:38,668 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786518327_0001_00_000002, stoppedTime=1452786518668, exitStatus=0
2016-01-14 15:48:38,668 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:38,669 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:38,669 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786518327_0001_1
2016-01-14 15:48:38,669 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786518327_0001_1
2016-01-14 15:48:38,670 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786518327_0001_1_00 [EF1ACD2574CE4C67966F6DB70D92535D]
2016-01-14 15:48:38,670 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9DC90CBA103E4575BC5169E30B287DA7 for vertex: vertex_1452786518327_0001_1_01 [B720CBD51C0C48DF97E86274C0E700E4]
2016-01-14 15:48:38,667 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:38,671 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=452984832, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786518653, Process_Duration=7, Process_End_Time=1452786518660, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:38,671 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:38,671 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,672 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518327_0001_00_000002
2016-01-14 15:48:38,672 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786518327_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:38,672 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518327_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:38,672 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786518327_0001_00_000002, Exiting
2016-01-14 15:48:38,672 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786518327_0001_00_000002
2016-01-14 15:48:38,672 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786518327_0001_00_000002
2016-01-14 15:48:38,673 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786518327_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:38,673 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786518327_0001_00_000002 completed successfully
2016-01-14 15:48:38,671 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:38,681 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786518327_0001_1, queueSize=0
2016-01-14 15:48:38,682 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:38,682 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518327_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786518327_0001_1, startTime=1452786518507, finishTime=1452786518680, timeTaken=173, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=905969664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573037241, Process_Duration=11, Process_End_Time=2905573037252, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:38,682 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786518327_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:38,683 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518327_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:38,683 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786518327_0001_00_000002
2016-01-14 15:48:38,683 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518327_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:38 Completed Dag: dag_1452786518327_0001_1
2016-01-14 15:48:38,684 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786518327_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:38,684 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786518327_0001_1_post
2016-01-14 15:48:38,685 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:38,685 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:38,688 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...e/false_false_false_false, applicationId=application_1452786518327_0001
2016-01-14 15:48:38,692 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:38,692 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:38,693 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:38,693 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:38,693 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:38,693 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:38,694 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:38,694 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:38,694 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:38,694 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:38,694 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:38,695 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:38,695 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:38,700 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E0CC18DB3C3746FF9EECC1840872818A/EF6B83F685C841AE95DA3580812FB7B5/.tez/application_1452786518327_0001
2016-01-14 15:48:38,701 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  completed in: 00:00.391
2016-01-14 15:48:38,760 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:38,760 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:38,793 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 15:48:38,809 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-01-14 15:48:38,809 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:38,810 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:38,810 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:38,829 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:38,841 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false
2016-01-14 15:48:38,898 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:38,898 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:38,899 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"]
2016-01-14 15:48:38,899 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  parallel execution of steps is enabled: false
2016-01-14 15:48:38,899 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  executing total steps: 1
2016-01-14 15:48:38,900 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  allocating management threads: 1
2016-01-14 15:48:38,900 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ve/true_false_false_false
2016-01-14 15:48:38,901 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:38,909 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:38,913 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:38,914 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:38,914 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:38,923 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001 doesn't exist and is created
2016-01-14 15:48:38,939 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:38,939 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:38,940 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001
2016-01-14 15:48:38,943 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786518914_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:38,950 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:38,951 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:38,953 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:38,953 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:38,953 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001/localmode-log-dir
2016-01-14 15:48:38,954 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001/localmode-log-dir/history.txt.appattempt_1452786518914_0001_000000, maxErrors=10
2016-01-14 15:48:38,954 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:38,954 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786518914_0001_000000, appSubmitTime=1452786518942, launchTime=1452786518942
2016-01-14 15:48:38,959 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:38,959 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:38,960 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:38,960 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:38,963 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39006
2016-01-14 15:48:38,965 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:38,968 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786518914_0001_000000, startTime=1452786518968
2016-01-14 15:48:38,968 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:38,968 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:39,040 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:39,040 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:39,040 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1452786518914_0001, dagName=[DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-01-14 15:48:39,056 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-01-14 15:48:39,057 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786518914_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001/localmode-log-dir/dag_1452786518914_0001_1.dot
2016-01-14 15:48:39,058 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001/localmode-log-dir/dag_1452786518914_0001_1-tez-dag.pb.txt
2016-01-14 15:48:39,069 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786518914_0001_1
2016-01-14 15:48:39,069 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:39,069 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-01-14 15:48:39 Running Dag: dag_1452786518914_0001_1
2016-01-14 15:48:39,084 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786518914_0001_1, submitTime=1452786519056
2016-01-14 15:48:39,084 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:39,085 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,085 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:39,085 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786518914_0001_1, initTime=1452786519084
2016-01-14 15:48:39,085 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:39,086 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518914_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:39,086 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_STARTED]: dagID=dag_1452786518914_0001_1, startTime=1452786519086
2016-01-14 15:48:39,086 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:39,087 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518914_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:39,087 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E4B20FBCBCE94A4EA6E4C85655610121 : {9806DA84762545DC85FAC9EACDBCED3B={InputName=9806DA84762545DC85FAC9EACDBCED3B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:39,087 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9806DA84762545DC85FAC9EACDBCED3B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:39,087 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,086 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:39,087 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,088 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,088 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]. Starting root input initializers: 1
2016-01-14 15:48:39,088 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:39,089 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,089 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9806DA84762545DC85FAC9EACDBCED3B on vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,091 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:39,091 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9806DA84762545DC85FAC9EACDBCED3B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:39,091 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:39,092 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:39,094 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:39,094 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9806DA84762545DC85FAC9EACDBCED3B on vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,097 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:39,097 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,097 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,098 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,098 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D392D21B56624D1F8E3194920657D6D3, vertexId=vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:39,098 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D392D21B56624D1F8E3194920657D6D3, vertexId=vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,101 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1452786518914_0001, dagName=[DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false
2016-01-14 15:48:39,102 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D392D21B56624D1F8E3194920657D6D3, vertexName=C43E0A9295CB4D9B8303357E015B7140, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:39,103 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D392D21B56624D1F8E3194920657D6D3, vertexId=vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:39,105 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:39,105 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, vertexId=vertex_1452786518914_0001_1_01, initRequestedTime=1452786519089, initedTime=1452786519104, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:39,106 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:39,106 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:39,106 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9806DA84762545DC85FAC9EACDBCED3B=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:39,107 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] parallelism set to 1
2016-01-14 15:48:39,107 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:39,108 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786518914_0001_1_00, eventCount=1
2016-01-14 15:48:39,109 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:39,110 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, vertexId=vertex_1452786518914_0001_1_00, initRequestedTime=1452786519087, initedTime=1452786519109, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:39,110 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] with distanceFromRoot: 0
2016-01-14 15:48:39,111 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:39,111 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:39,112 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,112 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:39,113 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786518914_0001_1_00, startRequestedTime=1452786519111, startedTime=1452786519111
2016-01-14 15:48:39,113 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:39,114 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:39,114 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_STARTED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, taskId=task_1452786518914_0001_1_00_000000, scheduledTime=1452786519114, launchTime=1452786519114
2016-01-14 15:48:39,115 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:39,115 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786518914_0001_1_00 for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] numStartedSources: 1 numSources: 1
2016-01-14 15:48:39,116 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] with distanceFromRoot: 1
2016-01-14 15:48:39,116 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:39,117 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E4B20FBCBCE94A4EA6E4C85655610121 in vertex: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,117 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C43E0A9295CB4D9B8303357E015B7140 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:39,118 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:39,118 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786518914_0001_1_01, startRequestedTime=1452786519117, startedTime=1452786519117
2016-01-14 15:48:39,118 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:39,119 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: E4B20FBCBCE94A4EA6E4C85655610121, VertexParallelism: 1, TaskAttemptID:attempt_1452786518914_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9806DA84762545DC85FAC9EACDBCED3B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C43E0A9295CB4D9B8303357E015B7140, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:39,119 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:39,119 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:39,120 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:39,120 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:39,120 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:39,120 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786518914_0001, with dag id: dag_1452786518914_0001_1
2016-01-14 15:48:39,121 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786518914_0001_00_000001
2016-01-14 15:48:39,122 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:39,124 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786518914_0001_00_000001
2016-01-14 15:48:39,124 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:39,125 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:39,125 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:39,126 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786518914_0001_00_000001, launchTime=1452786519125
2016-01-14 15:48:39,127 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518914_0001_00_000001
2016-01-14 15:48:39,127 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786518914_0001_1_00_000000_0] to container: [container_1452786518914_0001_00_000001]
2016-01-14 15:48:39,127 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:39,128 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786518914_0001_00_000001 given task: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,128 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518914_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,128 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786518914_0001_1_00_000000_0] started. Is using containerId: [container_1452786518914_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:39,128 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0, startTime=1452786519128, containerId=container_1452786518914_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786518914_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:39,128 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:39,129 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:39,129 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:39 Starting to run new task attempt: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,129 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,129 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:39,130 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:39,130 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:39,130 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:39,130 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:39,130 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:39,130 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: E4B20FBCBCE94A4EA6E4C85655610121, VertexParallelism: 1, TaskAttemptID:attempt_1452786518914_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9806DA84762545DC85FAC9EACDBCED3B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C43E0A9295CB4D9B8303357E015B7140, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:39,132 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:39,133 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,133 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:39,133 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786518914_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:39,133 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:39,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9806DA84762545DC85FAC9EACDBCED3B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:39,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C43E0A9295CB4D9B8303357E015B7140, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:39,134 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:39,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:39,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:39,135 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,136 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,136 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,140 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:39,141 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:39,141 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,141 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,140 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:39,143 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,143 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:39,148 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4B20FBCBCE94A4EA6E4C85655610121, ordinal: 0
2016-01-14 15:48:39,148 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4B20FBCBCE94A4EA6E4C85655610121, mem on start (mb), free: 105, total: 432, max: 672
2016-01-14 15:48:39,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:39,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:39,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:39,148 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:39,148 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:39,148 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:39,149 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:39,149 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:39,149 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:39,149 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:39,150 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9806DA84762545DC85FAC9EACDBCED3B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:39,150 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C43E0A9295CB4D9B8303357E015B7140, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:39,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:39,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9806DA84762545DC85FAC9EACDBCED3B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:39,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:39,151 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:39,150 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,151 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:39,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,152 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:39,152 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,155 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:39,156 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 9806DA84762545DC85FAC9EACDBCED3B
2016-01-14 15:48:39,156 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 1A8E1308617942248E56FF4F126F68B8
2016-01-14 15:48:39,157 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 1A8E1308617942248E56FF4F126F68B8
2016-01-14 15:48:39,157 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:39,158 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5a3c86c8
2016-01-14 15:48:39,158 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5a3c86c8
2016-01-14 15:48:39,159 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:39,159 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:39,160 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@3bb4e2c0; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6092c9df; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@45038101; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:39,160 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:39,161 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:39,161 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:39,183 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:39,184 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:39,184 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:39,184 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:39,184 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:39,185 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:39,185 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:39,185 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4B20FBCBCE94A4EA6E4C85655610121, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:39,189 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:39,190 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4B20FBCBCE94A4EA6E4C85655610121, mem on close (mb), free: 192, total: 422, max: 672
2016-01-14 15:48:39,190 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:39,190 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:39,190 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:39,190 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:40,327 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:40,327 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:40,896 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:40,896 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:41,550 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:41 Completed running task attempt: attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:41,553 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:41,553 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786519185, Process_Duration=4, Process_End_Time=1452786519189, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:41,553 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786518914_0001_1_00, eventCount=1
2016-01-14 15:48:41,554 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, taskAttemptId=attempt_1452786518914_0001_1_00_000000_0, startTime=1452786519128, finishTime=1452786521554, timeTaken=2426, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786519185, Process_Duration=4, Process_End_Time=1452786519189, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:41,554 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:41,554 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:41,554 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786518914_0001_1_00_000000_0
2016-01-14 15:48:41,555 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:41,555 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_FINISHED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, taskId=task_1452786518914_0001_1_00_000000, startTime=1452786519128, finishTime=1452786521554, timeTaken=2426, status=SUCCEEDED, successfulAttemptID=attempt_1452786518914_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786519185, Process_Duration=4, Process_End_Time=1452786519189, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:41,555 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:41,555 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:41,555 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] : 1
2016-01-14 15:48:41,555 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:41,556 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_FINISHED]: vertexName=E4B20FBCBCE94A4EA6E4C85655610121, vertexId=vertex_1452786518914_0001_1_00, initRequestedTime=1452786519087, initedTime=1452786519109, startRequestedTime=1452786519111, startedTime=1452786519111, finishTime=1452786521555, timeTaken=2444, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786519185, Process_Duration=4, Process_End_Time=1452786519189, Read_Duration=2, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786519128, firstTasksToStart=[ task_1452786518914_0001_1_00_000000 ], lastTaskFinishTime=1452786521554, lastTasksToFinish=[ task_1452786518914_0001_1_00_000000 ], minTaskDuration=2426, maxTaskDuration=2426, avgTaskDuration=2426.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786518914_0001_1_00_000000 ], longestDurationTasks=[ task_1452786518914_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:41,557 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:41,557 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:41,557 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] attempt: attempt_1452786518914_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:41,557 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:41,557 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:41,557 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C43E0A9295CB4D9B8303357E015B7140
2016-01-14 15:48:41,557 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,558 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:41,558 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:41,558 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786518914_0001_00_000001, stoppedTime=1452786521558, exitStatus=0
2016-01-14 15:48:41,559 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:41,559 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_STARTED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, taskId=task_1452786518914_0001_1_01_000000, scheduledTime=1452786521559, launchTime=1452786521559
2016-01-14 15:48:41,559 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:41,559 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:41,559 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786518914_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:41,559 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: C43E0A9295CB4D9B8303357E015B7140, VertexParallelism: 1, TaskAttemptID:attempt_1452786518914_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4B20FBCBCE94A4EA6E4C85655610121, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D392D21B56624D1F8E3194920657D6D3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:41,559 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:41,560 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:41,560 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:41,560 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:41,558 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518914_0001_00_000001
2016-01-14 15:48:41,560 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:41,561 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786518914_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:41,561 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518914_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:41,561 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786518914_0001_00_000002
2016-01-14 15:48:41,562 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786518914_0001_00_000001, Exiting
2016-01-14 15:48:41,562 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786518914_0001_00_000001
2016-01-14 15:48:41,562 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786518914_0001_00_000001
2016-01-14 15:48:41,562 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:41,564 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786518914_0001_00_000002
2016-01-14 15:48:41,564 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786518914_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:41,564 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786518914_0001_00_000001 completed successfully
2016-01-14 15:48:41,565 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:41,565 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 15:48:41,567 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:41,567 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786518914_0001_00_000002, launchTime=1452786521567
2016-01-14 15:48:41,567 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:41,568 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518914_0001_00_000002
2016-01-14 15:48:41,568 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786518914_0001_1_01_000000_0] to container: [container_1452786518914_0001_00_000002]
2016-01-14 15:48:41,568 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:41,569 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786518914_0001_00_000002 given task: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,569 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518914_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,569 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786518914_0001_1_01_000000_0] started. Is using containerId: [container_1452786518914_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:41 Starting to run new task attempt: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,569 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,569 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,569 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:41,570 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:41,570 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:41,570 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:41,570 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:41,570 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:41,570 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0, startTime=1452786521569, containerId=container_1452786518914_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786518914_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:41,570 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926] comprehensivesort/(1/1) ...ve/true_false_false_false, VertexName: C43E0A9295CB4D9B8303357E015B7140, VertexParallelism: 1, TaskAttemptID:attempt_1452786518914_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E4B20FBCBCE94A4EA6E4C85655610121, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D392D21B56624D1F8E3194920657D6D3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:41,570 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:41,571 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:41,571 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:41,571 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,571 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:41,571 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:41,572 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:41,573 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786518914_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:41,573 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D392D21B56624D1F8E3194920657D6D3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:41,574 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E4B20FBCBCE94A4EA6E4C85655610121, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:41,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:41,575 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D392D21B56624D1F8E3194920657D6D3
2016-01-14 15:48:41,575 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:41,575 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D392D21B56624D1F8E3194920657D6D3
2016-01-14 15:48:41,576 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:41,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:41,576 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,579 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:41,579 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:41,579 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,583 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,584 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C43E0A9295CB4D9B8303357E015B7140, ordinal: 1
2016-01-14 15:48:41,584 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C43E0A9295CB4D9B8303357E015B7140, mem on start (mb), free: 230, total: 444, max: 672
2016-01-14 15:48:41,584 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:41,585 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:41,585 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:41,589 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:41,589 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D392D21B56624D1F8E3194920657D6D3
2016-01-14 15:48:41,590 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:41,590 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:41,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:41,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:41,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:41,590 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:41,591 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:41,591 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D392D21B56624D1F8E3194920657D6D3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:41,591 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E4B20FBCBCE94A4EA6E4C85655610121, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:41,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:41,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E4B20FBCBCE94A4EA6E4C85655610121 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:41,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:41,591 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:41,593 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:41,593 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:41,593 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E4B20FBCBCE94A4EA6E4C85655610121 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:41,594 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:41,594 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:41,594 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:41,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,595 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:41,595 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,598 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:41,598 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786518914_0001_1_00_000000_0_10053, runDuration: 2415393, ]
2016-01-14 15:48:41,598 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 1A8E1308617942248E56FF4F126F68B8
2016-01-14 15:48:41,599 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"], id: D392D21B56624D1F8E3194920657D6D3
2016-01-14 15:48:41,599 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_false_false_false"]
2016-01-14 15:48:41,599 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 1A8E1308617942248E56FF4F126F68B8, for 1 inputs
2016-01-14 15:48:41,599 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786518914_0001_1_00_000000_0_10053] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786521599, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 15:48:41,600 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,600 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:41,600 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E4B20FBCBCE94A4EA6E4C85655610121] #10 in 1ms
2016-01-14 15:48:41,600 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E4B20FBCBCE94A4EA6E4C85655610121] #10, status:false, isInterrupted:false
2016-01-14 15:48:41,600 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:41,601 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:41,601 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:41,601 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:41,601 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:41,601 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,602 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:41,602 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C43E0A9295CB4D9B8303357E015B7140, all 1 inputs ready in: 00:00:00.002
2016-01-14 15:48:41,603 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@7b414197; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@79395fb8
2016-01-14 15:48:41,607 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:41,607 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:41,607 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,608 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786518914_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:41,608 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,608 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C43E0A9295CB4D9B8303357E015B7140, mem on close (mb), free: 226, total: 444, max: 672
2016-01-14 15:48:41,609 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E4B20FBCBCE94A4EA6E4C85655610121
2016-01-14 15:48:41,609 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:41,609 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:41,610 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:41,610 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:41,610 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:41 Completed running task attempt: attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,611 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:41,612 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, taskAttemptId=attempt_1452786518914_0001_1_01_000000_0, startTime=1452786521569, finishTime=1452786521612, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786521602, Process_Duration=6, Process_End_Time=1452786521608, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:41,612 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786518914_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:41,612 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786518914_0001_1_01_000000_0
2016-01-14 15:48:41,612 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:41,613 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:TASK_FINISHED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, taskId=task_1452786518914_0001_1_01_000000, startTime=1452786521569, finishTime=1452786521612, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452786518914_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786521602, Process_Duration=6, Process_End_Time=1452786521608, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:41,613 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786518914_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:41,613 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] : 1
2016-01-14 15:48:41,613 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:41,613 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:41,613 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:41,614 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:41,614 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:VERTEX_FINISHED]: vertexName=C43E0A9295CB4D9B8303357E015B7140, vertexId=vertex_1452786518914_0001_1_01, initRequestedTime=1452786519089, initedTime=1452786519104, startRequestedTime=1452786519117, startedTime=1452786519117, finishTime=1452786521613, timeTaken=2496, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786521602, Process_Duration=6, Process_End_Time=1452786521608, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786521569, firstTasksToStart=[ task_1452786518914_0001_1_01_000000 ], lastTaskFinishTime=1452786521612, lastTasksToFinish=[ task_1452786518914_0001_1_01_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786518914_0001_1_01_000000 ], longestDurationTasks=[ task_1452786518914_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:41,614 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:41,615 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:41,615 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:41,615 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786518914_0001_00_000002, stoppedTime=1452786521615, exitStatus=0
2016-01-14 15:48:41,615 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:41,615 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:41,615 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786518914_0001_1
2016-01-14 15:48:41,616 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786518914_0001_1
2016-01-14 15:48:41,616 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786518914_0001_1_00 [E4B20FBCBCE94A4EA6E4C85655610121]
2016-01-14 15:48:41,616 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D392D21B56624D1F8E3194920657D6D3 for vertex: vertex_1452786518914_0001_1_01 [C43E0A9295CB4D9B8303357E015B7140]
2016-01-14 15:48:41,619 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=754, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=465567744, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786521602, Process_Duration=6, Process_End_Time=1452786521608, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:41,619 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:41,619 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,620 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786518914_0001_00_000002
2016-01-14 15:48:41,620 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786518914_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:41,620 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786518914_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:41,621 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786518914_0001_00_000002, Exiting
2016-01-14 15:48:41,621 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786518914_0001_00_000002
2016-01-14 15:48:41,621 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786518914_0001_00_000002
2016-01-14 15:48:41,623 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786518914_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:41,624 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786518914_0001_00_000002 completed successfully
2016-01-14 15:48:41,630 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786518914_0001_1, queueSize=0
2016-01-14 15:48:41,630 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:41,630 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786518914_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786518914_0001_1, startTime=1452786519086, finishTime=1452786521628, timeTaken=2542, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=1518, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=355, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=931135488, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573040787, Process_Duration=10, Process_End_Time=2905573040797, Read_Duration=3, Tuples_Read=74, Tuples_Written=74, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:41,630 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786518914_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:41,631 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786518914_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:41,631 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786518914_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:41,631 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786518914_0001_00_000002
2016-01-14 15:48:41 Completed Dag: dag_1452786518914_0001_1
2016-01-14 15:48:41,632 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786518914_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:41,632 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786518914_0001_1_post
2016-01-14 15:48:41,633 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,633 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:41,636 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ve/true_false_false_false, applicationId=application_1452786518914_0001
2016-01-14 15:48:41,638 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:41,638 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:41,638 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:41,638 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:41,639 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:41,639 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:41,642 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:41,642 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:41,642 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:41,643 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:41,643 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:41,644 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:41,644 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:41,649 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/DEB663D1AADF4B74B95FDDBCAA430188/F5099F9B0EF548BA9AD5D0ABE8006926/.tez/application_1452786518914_0001
2016-01-14 15:48:41,649 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  completed in: 00:02.751
2016-01-14 15:48:41,703 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:41,703 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:41,742 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 15:48:41,750 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-01-14 15:48:41,750 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:41,751 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:41,751 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:41,762 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:41,770 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false
2016-01-14 15:48:41,806 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:41,807 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:41,807 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"]
2016-01-14 15:48:41,807 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  parallel execution of steps is enabled: false
2016-01-14 15:48:41,807 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  executing total steps: 1
2016-01-14 15:48:41,807 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  allocating management threads: 1
2016-01-14 15:48:41,808 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ive/true_true_false_false
2016-01-14 15:48:41,808 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:41,817 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:41,821 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:41,822 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:41,822 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:41,831 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001 doesn't exist and is created
2016-01-14 15:48:41,846 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:41,846 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:41,847 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001
2016-01-14 15:48:41,850 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786521822_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:41,857 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:41,858 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:41,860 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:41,860 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:41,860 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001/localmode-log-dir
2016-01-14 15:48:41,861 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001/localmode-log-dir/history.txt.appattempt_1452786521822_0001_000000, maxErrors=10
2016-01-14 15:48:41,861 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:41,862 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786521822_0001_000000, appSubmitTime=1452786521850, launchTime=1452786521850
2016-01-14 15:48:41,864 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:41,865 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:41,866 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:41,867 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:41,869 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53399
2016-01-14 15:48:41,870 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:41,872 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786521822_0001_000000, startTime=1452786521872
2016-01-14 15:48:41,872 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:41,872 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:41,947 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:41,947 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:41,947 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1452786521822_0001, dagName=[8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-01-14 15:48:41,960 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-01-14 15:48:41,961 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786521822_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001/localmode-log-dir/dag_1452786521822_0001_1.dot
2016-01-14 15:48:41,962 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001/localmode-log-dir/dag_1452786521822_0001_1-tez-dag.pb.txt
2016-01-14 15:48:41,970 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786521822_0001_1
2016-01-14 15:48:41,970 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:41,970 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-01-14 15:48:41 Running Dag: dag_1452786521822_0001_1
2016-01-14 15:48:41,983 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786521822_0001_1, submitTime=1452786521960
2016-01-14 15:48:41,983 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:41,983 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:41,984 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:41,985 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786521822_0001_1, initTime=1452786521983
2016-01-14 15:48:41,985 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786521822_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:41,985 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:41,985 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_STARTED]: dagID=dag_1452786521822_0001_1, startTime=1452786521985
2016-01-14 15:48:41,985 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:41,986 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:41,986 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786521822_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:41,986 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FF4F13E6FC2547F9A580FDA4118AB231 : {22A682D8507F41DA9000547AFD90768B={InputName=22A682D8507F41DA9000547AFD90768B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:41,986 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1452786521822_0001, dagName=[8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false
2016-01-14 15:48:41,986 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 22A682D8507F41DA9000547AFD90768B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:41,987 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,987 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,987 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,987 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]. Starting root input initializers: 1
2016-01-14 15:48:41,988 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:41,988 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,988 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 22A682D8507F41DA9000547AFD90768B on vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,990 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:41,990 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 22A682D8507F41DA9000547AFD90768B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:41,990 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:41,991 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:41,991 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:41,992 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,992 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,992 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,992 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=57285A20841E45FAB7A6A61E184394DF, vertexId=vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:41,993 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=57285A20841E45FAB7A6A61E184394DF, vertexId=vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,994 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:41,994 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 22A682D8507F41DA9000547AFD90768B on vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:41,995 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=57285A20841E45FAB7A6A61E184394DF, vertexName=BF55D45B5B2C43B7A40673361A13E1FC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:41,995 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=57285A20841E45FAB7A6A61E184394DF, vertexId=vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:41,996 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:41,996 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, vertexId=vertex_1452786521822_0001_1_01, initRequestedTime=1452786521988, initedTime=1452786521996, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:41,997 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:41,997 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:41,997 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {22A682D8507F41DA9000547AFD90768B=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:41,998 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] parallelism set to 1
2016-01-14 15:48:41,998 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:41,998 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786521822_0001_1_00, eventCount=1
2016-01-14 15:48:41,999 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, vertexId=vertex_1452786521822_0001_1_00, initRequestedTime=1452786521986, initedTime=1452786521999, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:41,999 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] with distanceFromRoot: 0
2016-01-14 15:48:41,999 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:41,999 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:42,000 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:42,000 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:42,001 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:42,000 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786521822_0001, with dag id: dag_1452786521822_0001_1
2016-01-14 15:48:42,001 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786521822_0001_1_00, startRequestedTime=1452786522000, startedTime=1452786522000
2016-01-14 15:48:42,002 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:42,002 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:42,003 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_STARTED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, taskId=task_1452786521822_0001_1_00_000000, scheduledTime=1452786522002, launchTime=1452786522002
2016-01-14 15:48:42,003 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:42,003 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786521822_0001_1_00 for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] numStartedSources: 1 numSources: 1
2016-01-14 15:48:42,003 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] with distanceFromRoot: 1
2016-01-14 15:48:42,004 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:42,004 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FF4F13E6FC2547F9A580FDA4118AB231 in vertex: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:42,004 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BF55D45B5B2C43B7A40673361A13E1FC with 1 source tasks and 1 pending tasks
2016-01-14 15:48:42,005 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:42,005 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786521822_0001_1_01, startRequestedTime=1452786522004, startedTime=1452786522004
2016-01-14 15:48:42,005 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:42,006 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: FF4F13E6FC2547F9A580FDA4118AB231, VertexParallelism: 1, TaskAttemptID:attempt_1452786521822_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=22A682D8507F41DA9000547AFD90768B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BF55D45B5B2C43B7A40673361A13E1FC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:42,007 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:42,007 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:42,009 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:42,009 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:42,010 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:42,010 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786521822_0001_00_000001
2016-01-14 15:48:42,010 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:42,012 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786521822_0001_00_000001
2016-01-14 15:48:42,012 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:42,012 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:42,013 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:42,013 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786521822_0001_00_000001, launchTime=1452786522013
2016-01-14 15:48:42,014 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786521822_0001_00_000001
2016-01-14 15:48:42,014 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786521822_0001_1_00_000000_0] to container: [container_1452786521822_0001_00_000001]
2016-01-14 15:48:42,015 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:42,015 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786521822_0001_00_000001 given task: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,015 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786521822_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42 Starting to run new task attempt: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,015 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786521822_0001_1_00_000000_0] started. Is using containerId: [container_1452786521822_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:42,016 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0, startTime=1452786522015, containerId=container_1452786521822_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786521822_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:42,016 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:42,016 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,016 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:42,016 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:42,016 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:42,017 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:42,017 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:42,017 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:42,017 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:42,018 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:42,018 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: FF4F13E6FC2547F9A580FDA4118AB231, VertexParallelism: 1, TaskAttemptID:attempt_1452786521822_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=22A682D8507F41DA9000547AFD90768B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BF55D45B5B2C43B7A40673361A13E1FC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:42,018 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:42,018 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:42,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:42,019 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786521822_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:42,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=22A682D8507F41DA9000547AFD90768B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:42,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BF55D45B5B2C43B7A40673361A13E1FC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:42,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:42,020 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:42,021 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,021 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,023 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:42,023 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,023 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:42,024 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:42,026 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:42,026 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:42,027 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:42,027 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:42,034 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF4F13E6FC2547F9A580FDA4118AB231, ordinal: 0
2016-01-14 15:48:42,034 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF4F13E6FC2547F9A580FDA4118AB231, mem on start (mb), free: 137, total: 444, max: 672
2016-01-14 15:48:42,034 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:42,034 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:42,035 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:42,035 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:42,035 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:42,035 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:42,035 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:42,035 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:42,036 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:42,036 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:42,036 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 22A682D8507F41DA9000547AFD90768B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:42,036 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BF55D45B5B2C43B7A40673361A13E1FC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:42,036 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:42,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,037 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:42,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 22A682D8507F41DA9000547AFD90768B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:42,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:42,037 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:42,038 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:42,038 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,038 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:42,038 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@19ede0f3
2016-01-14 15:48:42,039 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@19ede0f3
2016-01-14 15:48:42,040 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:42,040 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:42,041 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:42,042 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 22A682D8507F41DA9000547AFD90768B
2016-01-14 15:48:42,042 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 37048EF07AFD4E7090B99EC02EB63F37
2016-01-14 15:48:42,042 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 37048EF07AFD4E7090B99EC02EB63F37
2016-01-14 15:48:42,043 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@2fd492e8; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2cf8d5e0; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@4a17c31; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:42,044 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:42,044 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:42,045 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:42,064 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:42,064 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:42,064 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:42,064 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:42,065 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:42,065 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:42,065 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:42,066 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF4F13E6FC2547F9A580FDA4118AB231, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:42,070 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:42,070 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF4F13E6FC2547F9A580FDA4118AB231, mem on close (mb), free: 207, total: 444, max: 672
2016-01-14 15:48:42,070 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:42,070 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:42,070 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:42,071 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:43,134 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:43,135 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:43,693 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:43,694 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:44,967 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:44 Completed running task attempt: attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:44,972 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:44,972 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786521822_0001_1_00, eventCount=1
2016-01-14 15:48:44,973 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786522066, Process_Duration=4, Process_End_Time=1452786522070, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:44,973 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, taskAttemptId=attempt_1452786521822_0001_1_00_000000_0, startTime=1452786522015, finishTime=1452786524973, timeTaken=2958, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786522066, Process_Duration=4, Process_End_Time=1452786522070, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:44,973 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:44,974 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:44,973 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:44,974 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786521822_0001_1_00_000000_0
2016-01-14 15:48:44,974 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_FINISHED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, taskId=task_1452786521822_0001_1_00_000000, startTime=1452786522015, finishTime=1452786524974, timeTaken=2959, status=SUCCEEDED, successfulAttemptID=attempt_1452786521822_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786522066, Process_Duration=4, Process_End_Time=1452786522070, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:44,974 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:44,975 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786521822_0001_00_000001
2016-01-14 15:48:44,975 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:44,973 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:44,975 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:44,975 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] : 1
2016-01-14 15:48:44,976 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:44,977 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_FINISHED]: vertexName=FF4F13E6FC2547F9A580FDA4118AB231, vertexId=vertex_1452786521822_0001_1_00, initRequestedTime=1452786521986, initedTime=1452786521999, startRequestedTime=1452786522000, startedTime=1452786522000, finishTime=1452786524976, timeTaken=2976, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786522066, Process_Duration=4, Process_End_Time=1452786522070, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786522015, firstTasksToStart=[ task_1452786521822_0001_1_00_000000 ], lastTaskFinishTime=1452786524973, lastTasksToFinish=[ task_1452786521822_0001_1_00_000000 ], minTaskDuration=2958, maxTaskDuration=2958, avgTaskDuration=2958.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786521822_0001_1_00_000000 ], longestDurationTasks=[ task_1452786521822_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:44,977 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:44,977 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:44,977 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] attempt: attempt_1452786521822_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:44,978 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BF55D45B5B2C43B7A40673361A13E1FC
2016-01-14 15:48:44,978 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:44,978 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:44,978 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786521822_0001_00_000001, stoppedTime=1452786524978, exitStatus=0
2016-01-14 15:48:44,978 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:44,978 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:44,978 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:44,979 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_STARTED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, taskId=task_1452786521822_0001_1_01_000000, scheduledTime=1452786524979, launchTime=1452786524979
2016-01-14 15:48:44,979 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:44,979 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:44,980 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786521822_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:44,980 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:44,981 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:44,981 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: BF55D45B5B2C43B7A40673361A13E1FC, VertexParallelism: 1, TaskAttemptID:attempt_1452786521822_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF4F13E6FC2547F9A580FDA4118AB231, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=57285A20841E45FAB7A6A61E184394DF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:44,981 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:44,981 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:44,982 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786521822_0001_00_000002
2016-01-14 15:48:44,982 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:44,983 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786521822_0001_00_000002
2016-01-14 15:48:44,983 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:44,984 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:44,984 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:44,985 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786521822_0001_00_000002, launchTime=1452786524984
2016-01-14 15:48:44,985 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786521822_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 15:48:44,985 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786521822_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:44,985 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786521822_0001_00_000001, Exiting
2016-01-14 15:48:44,985 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786521822_0001_00_000001
2016-01-14 15:48:44,985 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786521822_0001_00_000001
2016-01-14 15:48:44,987 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786521822_0001_00_000002
2016-01-14 15:48:44,988 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786521822_0001_1_01_000000_0] to container: [container_1452786521822_0001_00_000002]
2016-01-14 15:48:44,988 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:44,988 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786521822_0001_00_000002 given task: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,989 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786521822_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,989 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786521822_0001_1_01_000000_0] started. Is using containerId: [container_1452786521822_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 15:48:44,989 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0, startTime=1452786524989, containerId=container_1452786521822_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786521822_0001_00_000002/teamcity, completedLogs=
2016-01-14 15:48:44,989 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:44,989 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:44 Starting to run new task attempt: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,989 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,990 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:44,989 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:44,990 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786521822_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:44,990 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:44,990 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786521822_0001_00_000001 completed successfully
2016-01-14 15:48:44,991 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 15:48:44,991 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:44,992 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:44,992 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:44,992 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:44,993 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A] comprehensivesort/(1/1) ...ive/true_true_false_false, VertexName: BF55D45B5B2C43B7A40673361A13E1FC, VertexParallelism: 1, TaskAttemptID:attempt_1452786521822_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FF4F13E6FC2547F9A580FDA4118AB231, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=57285A20841E45FAB7A6A61E184394DF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 15:48:44,993 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:44,993 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:44,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:44,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:44,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FF4F13E6FC2547F9A580FDA4118AB231, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 15:48:44,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=57285A20841E45FAB7A6A61E184394DF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 15:48:44,994 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:44,995 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:44,996 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:44,997 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:44,995 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786521822_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 15:48:44,998 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 15:48:44,998 INFO  orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 15:48:44,998 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:44,999 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:44,996 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:44,999 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 57285A20841E45FAB7A6A61E184394DF
2016-01-14 15:48:45,000 INFO  output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 15:48:45,000 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 57285A20841E45FAB7A6A61E184394DF
2016-01-14 15:48:45,006 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BF55D45B5B2C43B7A40673361A13E1FC, ordinal: 1
2016-01-14 15:48:45,006 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF55D45B5B2C43B7A40673361A13E1FC, mem on start (mb), free: 193, total: 447, max: 672
2016-01-14 15:48:45,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:45,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:45,007 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:45,012 INFO  output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 15:48:45,012 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 57285A20841E45FAB7A6A61E184394DF
2016-01-14 15:48:45,012 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:45,012 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:45,012 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 15:48:45,012 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 15:48:45,012 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:45,013 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:45,013 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 15:48:45,013 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FF4F13E6FC2547F9A580FDA4118AB231, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 15:48:45,013 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 57285A20841E45FAB7A6A61E184394DF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 15:48:45,013 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:45,013 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FF4F13E6FC2547F9A580FDA4118AB231 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:45,013 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,013 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:45,013 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:45,015 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:45,015 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 15:48:45,016 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FF4F13E6FC2547F9A580FDA4118AB231 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 15:48:45,016 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 15:48:45,016 INFO  orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 15:48:45,016 INFO  orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 15:48:45,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,019 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:45,019 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,020 INFO  orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452786521822_0001_1_00_000000_0_10059, runDuration: 2945228, ]
2016-01-14 15:48:45,022 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(comprehensivesort)[by:[{1}:'num']]
2016-01-14 15:48:45,022 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452786521822_0001_1_00_000000_0_10059] to DISK_DIRECT, CompressedSize=732, DecompressedSize=728,EndTime=1452786525022, TimeTaken=1, Rate=0.70 MB/s
2016-01-14 15:48:45,022 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']] streamed: true, id: 37048EF07AFD4E7090B99EC02EB63F37
2016-01-14 15:48:45,022 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"], id: 57285A20841E45FAB7A6A61E184394DF
2016-01-14 15:48:45,022 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,023 INFO  element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/true_true_false_false"]
2016-01-14 15:48:45,023 INFO  element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 37048EF07AFD4E7090B99EC02EB63F37, for 1 inputs
2016-01-14 15:48:45,023 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 15:48:45,024 INFO  orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FF4F13E6FC2547F9A580FDA4118AB231] #11 in 3ms
2016-01-14 15:48:45,024 INFO  orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FF4F13E6FC2547F9A580FDA4118AB231] #11, status:false, isInterrupted:false
2016-01-14 15:48:45,025 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 15:48:45,026 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 732 bytes from disk
2016-01-14 15:48:45,026 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:45,026 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 15:48:45,027 INFO  impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 712 bytes
2016-01-14 15:48:45,027 INFO  orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,027 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BF55D45B5B2C43B7A40673361A13E1FC, all 1 inputs ready in: 00:00:00.004
2016-01-14 15:48:45,027 INFO  orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 15:48:45,028 INFO  common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TuplePairDeserializer@15646798; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@72cd3720
2016-01-14 15:48:45,032 INFO  output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 15:48:45,032 INFO  output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 15:48:45,032 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,033 INFO  impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452786521822_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 15:48:45,033 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,033 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BF55D45B5B2C43B7A40673361A13E1FC, mem on close (mb), free: 189, total: 447, max: 672
2016-01-14 15:48:45,034 INFO  orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FF4F13E6FC2547F9A580FDA4118AB231
2016-01-14 15:48:45,034 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 15:48:45,035 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 15:48:45,035 INFO  orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 15:48:45,035 INFO  impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 15:48:45,035 INFO  impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 15:48:45 Completed running task attempt: attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,036 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:45,037 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, taskAttemptId=attempt_1452786521822_0001_1_01_000000_0, startTime=1452786524989, finishTime=1452786525037, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525027, Process_Duration=6, Process_End_Time=1452786525033, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:45,037 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786521822_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:45,038 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786521822_0001_1_01_000000_0
2016-01-14 15:48:45,038 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:45,038 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:45,038 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:TASK_FINISHED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, taskId=task_1452786521822_0001_1_01_000000, startTime=1452786524989, finishTime=1452786525038, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452786521822_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525027, Process_Duration=6, Process_End_Time=1452786525033, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37
2016-01-14 15:48:45,038 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786521822_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:45,038 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:45,039 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] : 1
2016-01-14 15:48:45,039 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:45,040 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:45,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:VERTEX_FINISHED]: vertexName=BF55D45B5B2C43B7A40673361A13E1FC, vertexId=vertex_1452786521822_0001_1_01, initRequestedTime=1452786521988, initedTime=1452786521996, startRequestedTime=1452786522004, startedTime=1452786522004, finishTime=1452786525039, timeTaken=3035, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525027, Process_Duration=6, Process_End_Time=1452786525033, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=37, vertexStats=firstTaskStartTime=1452786524989, firstTasksToStart=[ task_1452786521822_0001_1_01_000000 ], lastTaskFinishTime=1452786525037, lastTasksToFinish=[ task_1452786521822_0001_1_01_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786521822_0001_1_01_000000 ], longestDurationTasks=[ task_1452786521822_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:45,040 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:45,040 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:45,040 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786521822_0001_00_000002, stoppedTime=1452786525040, exitStatus=0
2016-01-14 15:48:45,041 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:45,041 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:45,041 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:45,041 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452786521822_0001_1
2016-01-14 15:48:45,041 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452786521822_0001_1
2016-01-14 15:48:45,041 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 57285A20841E45FAB7A6A61E184394DF for vertex: vertex_1452786521822_0001_1_01 [BF55D45B5B2C43B7A40673361A13E1FC]
2016-01-14 15:48:45,044 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=764, FILE_BYTES_WRITTEN=234, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=37, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=469237760, OUTPUT_RECORDS=37, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452786525027, Process_Duration=6, Process_End_Time=1452786525033, Read_Duration=1, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=37]]
2016-01-14 15:48:45,045 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:45,046 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:45,046 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786521822_0001_00_000002
2016-01-14 15:48:45,048 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452786521822_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 15:48:45,048 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786521822_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 15:48:45,049 INFO  task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452786521822_0001_00_000002, Exiting
2016-01-14 15:48:45,049 INFO  task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452786521822_0001_00_000002
2016-01-14 15:48:45,049 INFO  task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452786521822_0001_00_000002
2016-01-14 15:48:45,049 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452786521822_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 15:48:45,049 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452786521822_0001_00_000002 completed successfully
2016-01-14 15:48:45,053 INFO  impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452786521822_0001_1_00 [FF4F13E6FC2547F9A580FDA4118AB231]
2016-01-14 15:48:45,055 INFO  recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452786521822_0001_1, queueSize=0
2016-01-14 15:48:45,055 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 15:48:45,055 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786521822_0001_1][Event:DAG_FINISHED]: dagId=dag_1452786521822_0001_1, startTime=1452786521985, finishTime=1452786525053, timeTaken=3068, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=1128, FILE_BYTES_WRITTEN=998, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=12, REDUCE_INPUT_RECORDS=37, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=74, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=357, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=938475520, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=74, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=732, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=732, SHUFFLE_BYTES_DECOMPRESSED=728, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=732, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905573047093, Process_Duration=10, Process_End_Time=2905573047103, Read_Duration=1, Tuples_Read=74, Tuples_Written=74, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=37, Tuples_Written=37
2016-01-14 15:48:45,055 INFO  impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452786521822_0001_1 finished with state: SUCCEEDED
2016-01-14 15:48:45,055 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786521822_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 15:48:45,055 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786521822_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 15:48:45 Completed Dag: dag_1452786521822_0001_1
2016-01-14 15:48:45,056 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452786521822_0001_1, dagState=SUCCEEDED
2016-01-14 15:48:45,057 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786521822_0001_1_post
2016-01-14 15:48:45,057 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:45,057 INFO  app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 15:48:45,058 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452786521822_0001_00_000002
2016-01-14 15:48:45,061 INFO  client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ive/true_true_false_false, applicationId=application_1452786521822_0001
2016-01-14 15:48:45,063 INFO  client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 15:48:45,063 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 15:48:45,063 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 15:48:45,063 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 15:48:45,063 INFO  app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 15:48:45,063 INFO  app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 15:48:45,064 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 15:48:45,065 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 15:48:45,065 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 15:48:45,065 INFO  recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 15:48:45,066 INFO  recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 15:48:45,066 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 15:48:45,066 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 15:48:45,072 INFO  app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8CF25A26FA2742929437A79C81507FCD/ACD99DFCC9D84E8A9A78213AF80C999A/.tez/application_1452786521822_0001
2016-01-14 15:48:45,072 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  completed in: 00:03.266
2016-01-14 15:48:45,128 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 15:48:45,129 INFO  planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 15:48:45,165 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 15:48:45,170 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-01-14 15:48:45,170 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:45,170 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 15:48:45,170 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 15:48:45,182 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt
2016-01-14 15:48:45,190 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false
2016-01-14 15:48:45,227 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting
2016-01-14 15:48:45,228 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:45,228 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/sortedvaluesplatform/testComprehensiveGroupBy/comprehensive/false_true_false_false"]
2016-01-14 15:48:45,228 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  parallel execution of steps is enabled: false
2016-01-14 15:48:45,228 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  executing total steps: 1
2016-01-14 15:48:45,228 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort]  allocating management threads: 1
2016-01-14 15:48:45,230 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] starting step: (1/1) ...ve/false_true_false_false
2016-01-14 15:48:45,230 WARN  flow.Flow (BaseFlow.java:logWarn(1445)) - [comprehensivesort] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 15:48:45,240 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] tez session mode enabled: true
2016-01-14 15:48:45,244 INFO  client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:45,245 INFO  client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 15:48:45,245 INFO  client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since  'tez.ignore.lib.uris' is set to true
2016-01-14 15:48:45,253 INFO  client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001 doesn't exist and is created
2016-01-14 15:48:45,269 INFO  client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 15:48:45,269 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 15:48:45,270 INFO  client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001
2016-01-14 15:48:45,273 INFO  app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452786525245_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 15:48:45,282 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 15:48:45,283 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 15:48:45,285 INFO  node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 15:48:45,285 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 15:48:45,285 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001/localmode-log-dir
2016-01-14 15:48:45,286 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001/localmode-log-dir/history.txt.appattempt_1452786525245_0001_000000, maxErrors=10
2016-01-14 15:48:45,286 INFO  recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 15:48:45,286 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452786525245_0001_000000, appSubmitTime=1452786525273, launchTime=1452786525273
2016-01-14 15:48:45,290 INFO  history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 15:48:45,291 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 15:48:45,292 INFO  recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 15:48:45,292 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 15:48:45,293 INFO  client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41632
2016-01-14 15:48:45,298 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 15:48:45,300 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452786525245_0001_000000, startTime=1452786525300
2016-01-14 15:48:45,300 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 15:48:45,300 INFO  app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 15:48:45,369 INFO  client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 15:48:45,370 INFO  client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 15:48:45,370 INFO  client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ve/false_true_false_false, applicationId=application_1452786525245_0001, dagName=[3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-01-14 15:48:45,385 INFO  app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-01-14 15:48:45,386 INFO  app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452786525245_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001/localmode-log-dir/dag_1452786525245_0001_1.dot
2016-01-14 15:48:45,388 INFO  app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7/.tez/application_1452786525245_0001/localmode-log-dir/dag_1452786525245_0001_1-tez-dag.pb.txt
2016-01-14 15:48:45,396 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452786525245_0001_1
2016-01-14 15:48:45,397 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:45,397 INFO  app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-01-14 15:48:45 Running Dag: dag_1452786525245_0001_1
2016-01-14 15:48:45,410 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452786525245_0001_1, submitTime=1452786525385
2016-01-14 15:48:45,410 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 15:48:45,412 INFO  impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,412 INFO  impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 15:48:45,412 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452786525245_0001_1, initTime=1452786525411
2016-01-14 15:48:45,413 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786525245_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 15:48:45,413 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 15:48:45,414 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:DAG_STARTED]: dagID=dag_1452786525245_0001_1, startTime=1452786525414
2016-01-14 15:48:45,414 INFO  impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 15:48:45,414 INFO  impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452786525245_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 15:48:45,414 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F91DF1D91E3D44FD8E3DE03FB6680831 : {4659393DEC1445798DE07708DCE17AC1={InputName=4659393DEC1445798DE07708DCE17AC1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 15:48:45,414 INFO  impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4659393DEC1445798DE07708DCE17AC1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 15:48:45,415 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,414 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 15:48:45,415 INFO  client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ve/false_true_false_false, applicationId=application_1452786525245_0001, dagName=[3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false
2016-01-14 15:48:45,415 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,417 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,417 INFO  impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]. Starting root input initializers: 1
2016-01-14 15:48:45,417 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 15:48:45,417 INFO  impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,418 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4659393DEC1445798DE07708DCE17AC1 on vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,420 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 15:48:45,420 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 15:48:45,420 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,420 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4659393DEC1445798DE07708DCE17AC1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 15:48:45,421 INFO  impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,421 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,421 INFO  impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9839DB7B36AA49A697CD6E8731DD6EA6, vertexId=vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 15:48:45,421 INFO  common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 15:48:45,422 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 15:48:45,421 INFO  impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9839DB7B36AA49A697CD6E8731DD6EA6, vertexId=vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,424 INFO  committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9839DB7B36AA49A697CD6E8731DD6EA6, vertexName=CAA3E76E89A5451A89B58F6E4903A617, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 15:48:45,424 INFO  impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9839DB7B36AA49A697CD6E8731DD6EA6, vertexId=vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:45,425 INFO  hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 416
2016-01-14 15:48:45,425 INFO  dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4659393DEC1445798DE07708DCE17AC1 on vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,426 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:45,426 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAA3E76E89A5451A89B58F6E4903A617, vertexId=vertex_1452786525245_0001_1_01, initRequestedTime=1452786525417, initedTime=1452786525425, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 15:48:45,426 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] transitioned from NEW to INITED due to event V_INIT
2016-01-14 15:48:45,426 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:45,427 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4659393DEC1445798DE07708DCE17AC1=forAllWorkUnits=true, update=[1]}
2016-01-14 15:48:45,427 INFO  impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] parallelism set to 1
2016-01-14 15:48:45,427 INFO  impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 15:48:45,428 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786525245_0001_1_00, eventCount=1
2016-01-14 15:48:45,428 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 15:48:45,428 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, vertexId=vertex_1452786525245_0001_1_00, initRequestedTime=1452786525414, initedTime=1452786525428, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 15:48:45,429 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] with distanceFromRoot: 0
2016-01-14 15:48:45,429 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 15:48:45,429 INFO  impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F91DF1D91E3D44FD8E3DE03FB6680831
2016-01-14 15:48:45,429 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,430 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:45,430 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786525245_0001_1_00, startRequestedTime=1452786525429, startedTime=1452786525429
2016-01-14 15:48:45,430 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:45,430 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:45,431 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_STARTED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, taskId=task_1452786525245_0001_1_00_000000, scheduledTime=1452786525430, launchTime=1452786525430
2016-01-14 15:48:45,431 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:45,431 INFO  impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452786525245_0001_1_00 for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] numStartedSources: 1 numSources: 1
2016-01-14 15:48:45,431 INFO  impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] with distanceFromRoot: 1
2016-01-14 15:48:45,431 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:45,432 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F91DF1D91E3D44FD8E3DE03FB6680831 in vertex: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,432 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CAA3E76E89A5451A89B58F6E4903A617 with 1 source tasks and 1 pending tasks
2016-01-14 15:48:45,432 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 15:48:45,432 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452786525245_0001_1_01, startRequestedTime=1452786525432, startedTime=1452786525432
2016-01-14 15:48:45,432 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] transitioned from INITED to RUNNING due to event V_START
2016-01-14 15:48:45,433 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexName: F91DF1D91E3D44FD8E3DE03FB6680831, VertexParallelism: 1, TaskAttemptID:attempt_1452786525245_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4659393DEC1445798DE07708DCE17AC1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAA3E76E89A5451A89B58F6E4903A617, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:45,433 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 15:48:45,433 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 15:48:45,434 INFO  node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 15:48:45,434 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 15:48:45,434 INFO  container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 15:48:45,434 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452786525245_0001_00_000001
2016-01-14 15:48:45,435 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 15:48:45,436 INFO  task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier:  null, container_1452786525245_0001_00_000001
2016-01-14 15:48:45,436 WARN  impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-01-14 15:48:45,436 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 15:48:45,436 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [comprehensivesort] submitted tez dag to app master: application_1452786525245_0001, with dag id: dag_1452786525245_0001_1
2016-01-14 15:48:45,436 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452786525245_0001_00_000001, launchTime=1452786525436
2016-01-14 15:48:45,436 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 15:48:45,438 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786525245_0001_00_000001
2016-01-14 15:48:45,438 INFO  container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452786525245_0001_1_00_000000_0] to container: [container_1452786525245_0001_00_000001]
2016-01-14 15:48:45,439 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 15:48:45,439 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452786525245_0001_1_00_000000_0] started. Is using containerId: [container_1452786525245_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 15:48:45,439 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0, startTime=1452786525439, containerId=container_1452786525245_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452786525245_0001_00_000001/teamcity, completedLogs=
2016-01-14 15:48:45,439 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 15:48:45,439 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 15:48:45,440 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 15:48:45,440 INFO  app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452786525245_0001_00_000001 given task: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,440 INFO  task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452786525245_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45 Starting to run new task attempt: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,441 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,441 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:45,442 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 15:48:45,442 INFO  task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 15:48:45,442 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 15:48:45,442 INFO  task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 15:48:45,442 INFO  metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) -  Using ResourceCalculatorProcessTree : [ ]
2016-01-14 15:48:45,443 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexName: F91DF1D91E3D44FD8E3DE03FB6680831, VertexParallelism: 1, TaskAttemptID:attempt_1452786525245_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4659393DEC1445798DE07708DCE17AC1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAA3E76E89A5451A89B58F6E4903A617, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 15:48:45,443 INFO  resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 15:48:45,443 INFO  task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,443 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 15:48:45,443 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:45,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4659393DEC1445798DE07708DCE17AC1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 15:48:45,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CAA3E76E89A5451A89B58F6E4903A617, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 15:48:45,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 15:48:45,444 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:45,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,450 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,449 INFO  impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452786525245_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 15:48:45,449 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 15:48:45,451 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,452 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 15:48:45,452 INFO  input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 15:48:45,452 INFO  input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,453 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,453 INFO  impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 15:48:45,453 INFO  resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,454 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:45,459 INFO  tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F91DF1D91E3D44FD8E3DE03FB6680831, ordinal: 0
2016-01-14 15:48:45,459 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F91DF1D91E3D44FD8E3DE03FB6680831, mem on start (mb), free: 98, total: 447, max: 672
2016-01-14 15:48:45,459 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 15:48:45,459 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 15:48:45,460 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 15:48:45,460 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 15:48:45,460 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 15:48:45,460 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 15:48:45,460 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 15:48:45,460 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 15:48:45,460 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 15:48:45,461 INFO  resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 15:48:45,461 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4659393DEC1445798DE07708DCE17AC1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 15:48:45,461 INFO  resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CAA3E76E89A5451A89B58F6E4903A617, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 15:48:45,461 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 15:48:45,461 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4659393DEC1445798DE07708DCE17AC1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 15:48:45,461 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 15:48:45,461 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 15:48:45,462 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,462 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:45,462 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,462 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 15:48:45,462 INFO  task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,463 INFO  input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 15:48:45,463 INFO  input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@638bab94
2016-01-14 15:48:45,464 INFO  lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@638bab94
2016-01-14 15:48:45,464 INFO  input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 15:48:45,464 INFO  input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 15:48:45,466 INFO  graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:45,467 INFO  tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"] streamed: true, id: 4659393DEC1445798DE07708DCE17AC1
2016-01-14 15:48:45,467 INFO  tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']], id: 299EE9166C0A49B3A97712D62670F619
2016-01-14 15:48:45,468 INFO  element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(comprehensivesort)[by:[{1}:'num']] 299EE9166C0A49B3A97712D62670F619
2016-01-14 15:48:45,469 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TuplePairSerializer@12f61ade; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5b5dee8b; comparator=cascading.tuple.hadoop.util.GroupingSortingComparator@73d81689; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 15:48:45,470 INFO  impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.GroupingSortingPartitioner]
2016-01-14 15:48:45,470 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.GroupingSortingPartitioner
2016-01-14 15:48:45,470 INFO  common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 15:48:45,492 INFO  dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 15:48:45,492 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 15:48:45,492 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 15:48:45,492 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 15:48:45,493 INFO  dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 15:48:45,493 INFO  element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.4.x-tez-0.6.x/build/classes/test/data/lhs+rhs-cross.txt"]
2016-01-14 15:48:45,493 INFO  input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 15:48:45,494 INFO  tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F91DF1D91E3D44FD8E3DE03FB6680831, all 1 inputs ready in: 00:00:00.000
2016-01-14 15:48:45,498 INFO  task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:45,498 INFO  tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F91DF1D91E3D44FD8E3DE03FB6680831, mem on close (mb), free: 208, total: 451, max: 672
2016-01-14 15:48:45,498 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 15:48:45,498 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 15:48:45,499 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2016-01-14 15:48:45,499 INFO  dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2016-01-14 15:48:46,643 INFO  app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 15:48:46,643 INFO  app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 15:48:47,001 INFO  dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 15:48:47 Completed running task attempt: attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:47,004 INFO  task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 15:48:47,005 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452786525245_0001_1_00, eventCount=1
2016-01-14 15:48:47,005 INFO  runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=37, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452786525494, Process_Duration=4, Process_End_Time=1452786525498, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=37]]
2016-01-14 15:48:47,005 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 15:48:47,005 INFO  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: 
2016-01-14 15:48:47,005 WARN  common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 15:48:47,006 INFO  task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452786525245_0001_00_000001
2016-01-14 15:48:47,005 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, taskAttemptId=attempt_1452786525245_0001_1_00_000000_0, startTime=1452786525439, finishTime=1452786527005, timeTaken=1566, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525494, Process_Duration=4, Process_End_Time=1452786525498, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:47,006 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452786525245_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 15:48:47,006 INFO  rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 15:48:47,006 INFO  impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452786525245_0001_1_00_000000_0
2016-01-14 15:48:47,007 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_FINISHED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, taskId=task_1452786525245_0001_1_00_000000, startTime=1452786525439, finishTime=1452786527007, timeTaken=1568, status=SUCCEEDED, successfulAttemptID=attempt_1452786525245_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525494, Process_Duration=4, Process_End_Time=1452786525498, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37
2016-01-14 15:48:47,007 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 15:48:47,007 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 15:48:47,008 INFO  impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] : 1
2016-01-14 15:48:47,008 INFO  impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831]
2016-01-14 15:48:47,009 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 15:48:47,009 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:VERTEX_FINISHED]: vertexName=F91DF1D91E3D44FD8E3DE03FB6680831, vertexId=vertex_1452786525245_0001_1_00, initRequestedTime=1452786525414, initedTime=1452786525428, startRequestedTime=1452786525429, startedTime=1452786525429, finishTime=1452786527008, timeTaken=1579, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=364, FILE_BYTES_WRITTEN=764, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=37, GC_TIME_MILLIS=333, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=480247808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=888, OUTPUT_BYTES_WITH_OVERHEAD=728, OUTPUT_BYTES_PHYSICAL=732, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452786525494, Process_Duration=4, Process_End_Time=1452786525498, Read_Duration=0, Tuples_Read=37, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=37, vertexStats=firstTaskStartTime=1452786525439, firstTasksToStart=[ task_1452786525245_0001_1_00_000000 ], lastTaskFinishTime=1452786527005, lastTasksToFinish=[ task_1452786525245_0001_1_00_000000 ], minTaskDuration=1566, maxTaskDuration=1566, avgTaskDuration=1566.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452786525245_0001_1_00_000000 ], longestDurationTasks=[ task_1452786525245_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 15:48:47,009 INFO  impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 15:48:47,009 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 15:48:47,009 INFO  impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617] attempt: attempt_1452786525245_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 15:48:47,010 INFO  vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CAA3E76E89A5451A89B58F6E4903A617
2016-01-14 15:48:47,010 INFO  impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:47,010 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452786525245_0001_00_000001, stoppedTime=1452786527010, exitStatus=0
2016-01-14 15:48:47,010 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 15:48:47,010 INFO  impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452786525245_0001_1_00 [F91DF1D91E3D44FD8E3DE03FB6680831] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 15:48:47,010 INFO  history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452786525245_0001_1][Event:TASK_STARTED]: vertexName=CAA3E76E89A5451A89B58F6E4903A617, taskId=task_1452786525245_0001_1_01_000000, scheduledTime=1452786527010, launchTime=1452786527010
2016-01-14 15:48:47,011 INFO  impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452786525245_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 15:48:47,011 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 15:48:47,011 INFO  impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 15:48:47,011 INFO  launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452786525245_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 15:48:47,011 INFO  impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452786525245_0001_1_01 [CAA3E76E89A5451A89B58F6E4903A617]
2016-01-14 15:48:47,011 INFO  container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452786525245_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 15:48:47,012 INFO  impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3779C6E4E6394CCBA2470D7EA8EAC6AB/B425661E16FD428C85766E27003021E7] comprehensivesort/(1/1) ...ve/false_true_false_false, VertexNa