2016-05-17 22:27:50,033 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 22:27:50,058 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.operation.assertion.AppliedAssertionsPlatformTest, with platform: hadoop2-tez
2016-05-17 22:27:50,093 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 22:27:50,445 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 22:27:50,815 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-17 22:27:51,060 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:51,062 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:51,180 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: D847978384E543D6969388ACF4047AD4
2016-05-17 22:27:51,417 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.100
2016-05-17 22:27:51,444 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.127
2016-05-17 22:27:51,445 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:51,446 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:51,447 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:51,575 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:51,731 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/STRICT
2016-05-17 22:27:51,940 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 22:27:51,942 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:51,942 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:51,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/STRICT"]
2016-05-17 22:27:51,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:51,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:51,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:51,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...Removal/value/NONE/STRICT
2016-05-17 22:27:51,945 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:51,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:52,154 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:52,160 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:52,160 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:52,205 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D/.tez/application_1463524072159_0001 doesn't exist and is created
2016-05-17 22:27:52,285 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:52,286 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:52,288 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D/.tez/application_1463524072159_0001
2016-05-17 22:27:52,331 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524072159_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:52,372 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:52,386 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 22:27:52,387 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 22:27:52,387 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:52,388 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:52,410 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:52,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:52,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D/.tez/application_1463524072159_0001/localmode-log-dir
2016-05-17 22:27:52,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D/.tez/application_1463524072159_0001/localmode-log-dir/history.txt.appattempt_1463524072159_0001_000000, maxErrors=10
2016-05-17 22:27:52,417 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:52,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524072159_0001_000000, appSubmitTime=1463524072312, launchTime=1463524072329
2016-05-17 22:27:52,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:52,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:52,428 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:52,428 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:52,484 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56085
2016-05-17 22:27:52,487 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 22:27:52,488 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 22:27:52,488 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:52,526 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:52,588 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 22:27:52,588 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 22:27:52,589 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:52,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:52,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524072159_0001_000000, startTime=1463524072615
2016-05-17 22:27:52,616 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:52,689 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:52,705 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:52,705 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...Removal/value/NONE/STRICT, applicationId=application_1463524072159_0001, dagName=[1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D] test/(1/1) ...Removal/value/NONE/STRICT
2016-05-17 22:27:52,939 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D] test/(1/1) ...Removal/value/NONE/STRICT
2016-05-17 22:27:53,038 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524072159_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D/.tez/application_1463524072159_0001/localmode-log-dir/dag_1463524072159_0001_1.dot
2016-05-17 22:27:53,043 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D/.tez/application_1463524072159_0001/localmode-log-dir/dag_1463524072159_0001_1-tez-dag.pb.txt
2016-05-17 22:27:53,148 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524072159_0001_1
2016-05-17 22:27:53,149 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:53,149 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D] test/(1/1) ...Removal/value/NONE/STRICT
2016-05-17 22:27:53 Running Dag: dag_1463524072159_0001_1
2016-05-17 22:27:53,177 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:27:53,177 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524072159_0001_1, submitTime=1463524072940
2016-05-17 22:27:53,207 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C30A2723A2BC42BB9824ACB0F97955FF
2016-05-17 22:27:53,215 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:27:53,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:27:53,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524072159_0001_1, initTime=1463524073179
2016-05-17 22:27:53,218 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524072159_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:27:53,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:DAG_STARTED]: dagID=dag_1463524072159_0001_1, startTime=1463524073219
2016-05-17 22:27:53,220 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:27:53,221 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:27:53,222 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524072159_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:27:53,222 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ED9A3B510D044416B3311A0EEA14313C : {F46F670477D84A0AAB06EAFEF7B05218={InputName=F46F670477D84A0AAB06EAFEF7B05218}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:27:53,223 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F46F670477D84A0AAB06EAFEF7B05218, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:27:53,223 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:53,229 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:53,229 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:53,229 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...Removal/value/NONE/STRICT, applicationId=application_1463524072159_0001, dagName=[1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D] test/(1/1) ...Removal/value/NONE/STRICT
2016-05-17 22:27:53,231 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]. Starting root input initializers: 1
2016-05-17 22:27:53,235 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F46F670477D84A0AAB06EAFEF7B05218 on vertex vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:53,247 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:27:53,249 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:53,255 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:27:53,257 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F46F670477D84A0AAB06EAFEF7B05218 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:27:53,258 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:27:53,259 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:27:53,277 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:27:53,277 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:53,294 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:53,296 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524072159_0001, with dag id: dag_1463524072159_0001_1
2016-05-17 22:27:53,298 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:53,299 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D6D9BA949B044B61A4FC5DED6303FD1A, vertexId=vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:27:53,302 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D6D9BA949B044B61A4FC5DED6303FD1A, vertexId=vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:53,312 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:27:53,316 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F46F670477D84A0AAB06EAFEF7B05218 on vertex vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:53,325 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D6D9BA949B044B61A4FC5DED6303FD1A, vertexName=C30A2723A2BC42BB9824ACB0F97955FF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:27:53,336 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D6D9BA949B044B61A4FC5DED6303FD1A, vertexId=vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:53,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C30A2723A2BC42BB9824ACB0F97955FF, vertexId=vertex_1463524072159_0001_1_01, initRequestedTime=1463524073247, initedTime=1463524073339, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:27:53,341 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:27:53,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:53,347 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:53,349 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F46F670477D84A0AAB06EAFEF7B05218=forAllWorkUnits=true, update=[1]}
2016-05-17 22:27:53,351 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] parallelism set to 1
2016-05-17 22:27:53,351 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:53,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524072159_0001_1_00, eventCount=1
2016-05-17 22:27:53,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:53,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ED9A3B510D044416B3311A0EEA14313C, vertexId=vertex_1463524072159_0001_1_00, initRequestedTime=1463524073222, initedTime=1463524073362, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:27:53,365 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] with distanceFromRoot: 0
2016-05-17 22:27:53,365 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:27:53,367 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:53,367 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:53,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:53,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524072159_0001_1_00, startRequestedTime=1463524073366, startedTime=1463524073366
2016-05-17 22:27:53,372 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:53,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:53,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:TASK_STARTED]: vertexName=ED9A3B510D044416B3311A0EEA14313C, taskId=task_1463524072159_0001_1_00_000000, scheduledTime=1463524073408, launchTime=1463524073408
2016-05-17 22:27:53,410 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524072159_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:53,414 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524072159_0001_1_00 for vertex: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF] numStartedSources: 1 numSources: 1
2016-05-17 22:27:53,414 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF] with distanceFromRoot: 1
2016-05-17 22:27:53,418 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:53,424 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ED9A3B510D044416B3311A0EEA14313C in vertex: C30A2723A2BC42BB9824ACB0F97955FF
2016-05-17 22:27:53,427 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C30A2723A2BC42BB9824ACB0F97955FF with 1 source tasks and 1 pending tasks
2016-05-17 22:27:53,428 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:53,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524072159_0001_1_01, startRequestedTime=1463524073418, startedTime=1463524073418
2016-05-17 22:27:53,428 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:53,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D] test/(1/1) ...Removal/value/NONE/STRICT, VertexName: ED9A3B510D044416B3311A0EEA14313C, VertexParallelism: 1, TaskAttemptID:attempt_1463524072159_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F46F670477D84A0AAB06EAFEF7B05218, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C30A2723A2BC42BB9824ACB0F97955FF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:53,435 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524072159_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:53,435 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:53,476 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:27:53,480 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:27:53,480 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:27:53,484 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524072159_0001_00_000001
2016-05-17 22:27:53,484 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:53,528 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524072159_0001_00_000001
2016-05-17 22:27:53,528 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:53,535 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:53,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:53,536 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524072159_0001_00_000001, launchTime=1463524073536
2016-05-17 22:27:53,539 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524072159_0001_00_000001
2016-05-17 22:27:53,542 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524072159_0001_1_00_000000_0] to container: [container_1463524072159_0001_00_000001]
2016-05-17 22:27:53,543 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:53,544 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524072159_0001_00_000001 given task: attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,544 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524072159_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,545 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524072159_0001_1_00_000000_0] started. Is using containerId: [container_1463524072159_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:27:53 Starting to run new task attempt: attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,546 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:53,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ED9A3B510D044416B3311A0EEA14313C, taskAttemptId=attempt_1463524072159_0001_1_00_000000_0, startTime=1463524073544, containerId=container_1463524072159_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524072159_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:27:53,550 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524072159_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:53,550 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 22:27:53,551 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:53,552 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:53,552 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:53,553 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:53,554 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524072159_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:53,555 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:53,578 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:53,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D] test/(1/1) ...Removal/value/NONE/STRICT, VertexName: ED9A3B510D044416B3311A0EEA14313C, VertexParallelism: 1, TaskAttemptID:attempt_1463524072159_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F46F670477D84A0AAB06EAFEF7B05218, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C30A2723A2BC42BB9824ACB0F97955FF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:53,581 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:53,586 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:53,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:53,587 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524072159_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:27:53,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F46F670477D84A0AAB06EAFEF7B05218, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:27:53,592 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:53,592 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C30A2723A2BC42BB9824ACB0F97955FF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:27:53,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:53,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F46F670477D84A0AAB06EAFEF7B05218
2016-05-17 22:27:53,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:53,597 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F46F670477D84A0AAB06EAFEF7B05218
2016-05-17 22:27:53,608 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C30A2723A2BC42BB9824ACB0F97955FF
2016-05-17 22:27:53,620 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:27:53,620 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:53,620 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F46F670477D84A0AAB06EAFEF7B05218
2016-05-17 22:27:53,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F46F670477D84A0AAB06EAFEF7B05218
2016-05-17 22:27:53,622 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:27:53,622 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C30A2723A2BC42BB9824ACB0F97955FF
2016-05-17 22:27:53,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C30A2723A2BC42BB9824ACB0F97955FF
2016-05-17 22:27:53,637 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ED9A3B510D044416B3311A0EEA14313C, ordinal: 0
2016-05-17 22:27:53,637 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ED9A3B510D044416B3311A0EEA14313C, mem on start (mb), free: 255, total: 281, max: 672
2016-05-17 22:27:53,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:53,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:53,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:53,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:53,639 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:53,643 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:27:53,644 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:53,644 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:27:53,644 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:53,645 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:27:53,645 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F46F670477D84A0AAB06EAFEF7B05218, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:27:53,645 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C30A2723A2BC42BB9824ACB0F97955FF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:27:53,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F46F670477D84A0AAB06EAFEF7B05218 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F46F670477D84A0AAB06EAFEF7B05218
2016-05-17 22:27:53,648 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F46F670477D84A0AAB06EAFEF7B05218
2016-05-17 22:27:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:53,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:53,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:53,650 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,652 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:27:53,662 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4f4eb05
2016-05-17 22:27:53,672 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4f4eb05
2016-05-17 22:27:53,673 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:27:53,673 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:27:53,683 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:53,710 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: F46F670477D84A0AAB06EAFEF7B05218
2016-05-17 22:27:53,710 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: F4692993C50241A593622116B6DA6B3B
2016-05-17 22:27:53,710 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] F4692993C50241A593622116B6DA6B3B
2016-05-17 22:27:53,740 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@52768d54; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6538b017; comparator=cascading.tuple.hadoop.util.TupleComparator@6cb0e9eb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:27:53,743 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:27:53,745 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:27:53,745 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:53,863 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:27:53,863 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:27:53,863 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:27:53,863 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:27:53,863 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:27:53,865 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:53,866 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:53,868 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ED9A3B510D044416B3311A0EEA14313C, all 1 inputs ready in: 00:00:00.002
2016-05-17 22:27:53,877 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,878 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ED9A3B510D044416B3311A0EEA14313C, mem on close (mb), free: 143, total: 281, max: 672
2016-05-17 22:27:53,878 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:27:53,878 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:27:53,878 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:27:53,878 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:27:53,894 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:27:53 Completed running task attempt: attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,912 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524072159_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:53,913 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524072159_0001_1_00, eventCount=1
2016-05-17 22:27:53,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:53,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524073868, Process_Duration=9, Process_End_Time=1463524073877, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:27:53,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ED9A3B510D044416B3311A0EEA14313C, taskAttemptId=attempt_1463524072159_0001_1_00_000000_0, startTime=1463524073544, finishTime=1463524073914, timeTaken=370, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524073868, Process_Duration=9, Process_End_Time=1463524073877, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:53,924 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524072159_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:53,924 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:53,925 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524072159_0001_1_00_000000_0
2016-05-17 22:27:53,924 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:53,926 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:53,928 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524072159_0001_00_000001
2016-05-17 22:27:53,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:53,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:TASK_FINISHED]: vertexName=ED9A3B510D044416B3311A0EEA14313C, taskId=task_1463524072159_0001_1_00_000000, startTime=1463524073544, finishTime=1463524073927, timeTaken=383, status=SUCCEEDED, successfulAttemptID=attempt_1463524072159_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524073868, Process_Duration=9, Process_End_Time=1463524073877, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:53,935 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524072159_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:53,936 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] : 1
2016-05-17 22:27:53,936 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:53,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:53,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:VERTEX_FINISHED]: vertexName=ED9A3B510D044416B3311A0EEA14313C, vertexId=vertex_1463524072159_0001_1_00, initRequestedTime=1463524073222, initedTime=1463524073362, startRequestedTime=1463524073366, startedTime=1463524073366, finishTime=1463524073937, timeTaken=571, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524073868, Process_Duration=9, Process_End_Time=1463524073877, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524073544, firstTasksToStart=[ task_1463524072159_0001_1_00_000000 ], lastTaskFinishTime=1463524073914, lastTasksToFinish=[ task_1463524072159_0001_1_00_000000 ], minTaskDuration=370, maxTaskDuration=370, avgTaskDuration=370.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524072159_0001_1_00_000000 ], longestDurationTasks=[ task_1463524072159_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:53,948 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:53,948 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:53,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:53,961 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524072159_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:27:53,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524072159_0001_00_000001, stoppedTime=1463524073958, exitStatus=0
2016-05-17 22:27:53,962 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524072159_0001_00_000001: 34 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:53,963 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524072159_0001_00_000001, Exiting
2016-05-17 22:27:53,963 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524072159_0001_00_000001
2016-05-17 22:27:53,963 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524072159_0001_00_000001
2016-05-17 22:27:53,963 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:53,967 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF] attempt: attempt_1463524072159_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:27:53,967 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C30A2723A2BC42BB9824ACB0F97955FF
2016-05-17 22:27:53,967 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:53,968 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524072159_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:53,969 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524072159_0001_00_000001 completed successfully
2016-05-17 22:27:53,969 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:53,969 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524072159_0001_00_000001
2016-05-17 22:27:53,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:TASK_STARTED]: vertexName=C30A2723A2BC42BB9824ACB0F97955FF, taskId=task_1463524072159_0001_1_01_000000, scheduledTime=1463524073969, launchTime=1463524073969
2016-05-17 22:27:53,970 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524072159_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:53,970 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:27:53,970 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:53,970 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:53,971 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:27:53,971 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D] test/(1/1) ...Removal/value/NONE/STRICT, VertexName: C30A2723A2BC42BB9824ACB0F97955FF, VertexParallelism: 1, TaskAttemptID:attempt_1463524072159_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ED9A3B510D044416B3311A0EEA14313C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D6D9BA949B044B61A4FC5DED6303FD1A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:53,972 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524072159_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:53,972 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:53,973 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524072159_0001_00_000002
2016-05-17 22:27:53,974 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:53,977 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524072159_0001_00_000002
2016-05-17 22:27:53,977 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:27:53,979 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:53,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524072159_0001_00_000002, launchTime=1463524073979
2016-05-17 22:27:53,979 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:53,980 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524072159_0001_00_000002
2016-05-17 22:27:53,981 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524072159_0001_1_01_000000_0] to container: [container_1463524072159_0001_00_000002]
2016-05-17 22:27:53,981 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:53,981 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524072159_0001_00_000002 given task: attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:53,982 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524072159_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:53,982 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524072159_0001_1_01_000000_0] started. Is using containerId: [container_1463524072159_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:27:53,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:53,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C30A2723A2BC42BB9824ACB0F97955FF, taskAttemptId=attempt_1463524072159_0001_1_01_000000_0, startTime=1463524073982, containerId=container_1463524072159_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524072159_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:27:53 Starting to run new task attempt: attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:53,984 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:53,984 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:53,985 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:53,985 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:53,985 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:53,985 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:53,986 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:53,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D] test/(1/1) ...Removal/value/NONE/STRICT, VertexName: C30A2723A2BC42BB9824ACB0F97955FF, VertexParallelism: 1, TaskAttemptID:attempt_1463524072159_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ED9A3B510D044416B3311A0EEA14313C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D6D9BA949B044B61A4FC5DED6303FD1A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:53,986 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:53,987 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524072159_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:53,987 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:53,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:53,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:53,988 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524072159_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:53,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:53,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D6D9BA949B044B61A4FC5DED6303FD1A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:27:53,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ED9A3B510D044416B3311A0EEA14313C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:27:53,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:53,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:53,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D6D9BA949B044B61A4FC5DED6303FD1A
2016-05-17 22:27:53,996 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:27:53,996 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D6D9BA949B044B61A4FC5DED6303FD1A
2016-05-17 22:27:53,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:54,001 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524072159_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:27:53,999 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:54,018 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:27:54,019 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:27:54,023 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:54,026 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C30A2723A2BC42BB9824ACB0F97955FF, ordinal: 1
2016-05-17 22:27:54,026 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C30A2723A2BC42BB9824ACB0F97955FF, mem on start (mb), free: 132, total: 281, max: 672
2016-05-17 22:27:54,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:54,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:54,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:54,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:54,041 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:27:54,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D6D9BA949B044B61A4FC5DED6303FD1A
2016-05-17 22:27:54,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:54,042 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:54,042 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:27:54,043 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:54,044 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:27:54,044 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:54,044 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:27:54,044 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D6D9BA949B044B61A4FC5DED6303FD1A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:27:54,044 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ED9A3B510D044416B3311A0EEA14313C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:27:54,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:54,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:54,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ED9A3B510D044416B3311A0EEA14313C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:54,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:54,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:54,055 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:54,056 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:27:54,061 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ED9A3B510D044416B3311A0EEA14313C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:27:54,064 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:27:54,064 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:27:54,071 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:27:54,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:54,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:54,075 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:54,076 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524072159_0001_1_00_000000_0_10003, runDuration: 289789, ]
2016-05-17 22:27:54,082 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:27:54,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524072159_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524074096, TimeTaken=7, Rate=0.09 MB/s
2016-05-17 22:27:54,099 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:54,100 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:27:54,100 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [ED9A3B510D044416B3311A0EEA14313C] #1 in 12ms
2016-05-17 22:27:54,102 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: F4692993C50241A593622116B6DA6B3B
2016-05-17 22:27:54,102 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ED9A3B510D044416B3311A0EEA14313C] #1, status:false, isInterrupted:false
2016-05-17 22:27:54,103 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/STRICT"], id: D6D9BA949B044B61A4FC5DED6303FD1A
2016-05-17 22:27:54,103 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:27:54,106 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:27:54,107 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:54,112 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/STRICT"]
2016-05-17 22:27:54,113 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] F4692993C50241A593622116B6DA6B3B, for 1 inputs
2016-05-17 22:27:54,114 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:27:54,118 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:27:54,119 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:54,120 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:27:54,121 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C30A2723A2BC42BB9824ACB0F97955FF, all 1 inputs ready in: 00:00:00.007
2016-05-17 22:27:54,127 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1777c33c; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@37b2a42d
2016-05-17 22:27:54,134 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:27:54,135 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:27:54,136 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:54,137 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524072159_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:27:54,138 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:54,138 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C30A2723A2BC42BB9824ACB0F97955FF, mem on close (mb), free: 124, total: 281, max: 672
2016-05-17 22:27:54,139 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ED9A3B510D044416B3311A0EEA14313C
2016-05-17 22:27:54,140 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:27:54,141 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:27:54,142 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:54,142 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:27:54,142 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:27:54 Completed running task attempt: attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:54,145 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524072159_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:54,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:54,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524074121, Process_Duration=17, Process_End_Time=1463524074138, Read_Duration=3, Tuples_Read=7, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=1]]
2016-05-17 22:27:54,150 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:54,150 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:54,151 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524072159_0001_00_000002
2016-05-17 22:27:54,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C30A2723A2BC42BB9824ACB0F97955FF, taskAttemptId=attempt_1463524072159_0001_1_01_000000_0, startTime=1463524073982, finishTime=1463524074147, timeTaken=165, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524074121, Process_Duration=17, Process_End_Time=1463524074138, Read_Duration=3, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:54,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524072159_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:54,152 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524072159_0001_1_01_000000_0
2016-05-17 22:27:54,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:54,153 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:54,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:TASK_FINISHED]: vertexName=C30A2723A2BC42BB9824ACB0F97955FF, taskId=task_1463524072159_0001_1_01_000000, startTime=1463524073982, finishTime=1463524074152, timeTaken=170, status=SUCCEEDED, successfulAttemptID=attempt_1463524072159_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524074121, Process_Duration=17, Process_End_Time=1463524074138, Read_Duration=3, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:54,155 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524072159_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:54,155 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF] : 1
2016-05-17 22:27:54,156 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:54,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:54,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:VERTEX_FINISHED]: vertexName=C30A2723A2BC42BB9824ACB0F97955FF, vertexId=vertex_1463524072159_0001_1_01, initRequestedTime=1463524073247, initedTime=1463524073339, startRequestedTime=1463524073418, startedTime=1463524073418, finishTime=1463524074156, timeTaken=738, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524074121, Process_Duration=17, Process_End_Time=1463524074138, Read_Duration=3, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1, vertexStats=firstTaskStartTime=1463524073982, firstTasksToStart=[ task_1463524072159_0001_1_01_000000 ], lastTaskFinishTime=1463524074147, lastTasksToFinish=[ task_1463524072159_0001_1_01_000000 ], minTaskDuration=165, maxTaskDuration=165, avgTaskDuration=165.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524072159_0001_1_01_000000 ], longestDurationTasks=[ task_1463524072159_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:54,160 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:54,160 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:54,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:54,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524072159_0001_00_000002, stoppedTime=1463524074160, exitStatus=0
2016-05-17 22:27:54,162 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:54,162 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:54,162 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524072159_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 22:27:54,163 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524072159_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:54,163 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524072159_0001_00_000002, Exiting
2016-05-17 22:27:54,163 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524072159_0001_00_000002
2016-05-17 22:27:54,163 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524072159_0001_00_000002
2016-05-17 22:27:54,162 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524072159_0001_1
2016-05-17 22:27:54,164 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524072159_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:54,165 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524072159_0001_00_000002 completed successfully
2016-05-17 22:27:54,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524072159_0001_1
2016-05-17 22:27:54,166 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D6D9BA949B044B61A4FC5DED6303FD1A for vertex: vertex_1463524072159_0001_1_01 [C30A2723A2BC42BB9824ACB0F97955FF]
2016-05-17 22:27:54,181 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463524072159_0001_1_00 [ED9A3B510D044416B3311A0EEA14313C]
2016-05-17 22:27:54,188 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524072159_0001_1, queueSize=0
2016-05-17 22:27:54,189 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:27:54,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524072159_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524072159_0001_1, startTime=1463524073219, finishTime=1463524074181, timeTaken=962, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2891, FILE_BYTES_WRITTEN=704, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=14, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=8, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=23, LAST_EVENT_RECEIVED=23, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927048147989, Process_Duration=26, Process_End_Time=2927048148015, Read_Duration=5, Tuples_Read=17, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=1
2016-05-17 22:27:54,190 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524072159_0001_1 finished with state: SUCCEEDED
2016-05-17 22:27:54,191 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524072159_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:27:54,193 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524072159_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:27:54,193 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524072159_0001_00_000002
2016-05-17 22:27:54 Completed Dag: dag_1463524072159_0001_1
2016-05-17 22:27:54,194 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524072159_0001_1, dagState=SUCCEEDED
2016-05-17 22:27:54,194 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524072159_0001_1_post
2016-05-17 22:27:54,194 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:54,194 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:27:54,202 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...Removal/value/NONE/STRICT, applicationId=application_1463524072159_0001
2016-05-17 22:27:54,206 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:27:54,206 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:27:54,206 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:27:54,206 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:27:54,206 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:27:54,208 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:27:54,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:27:54,209 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:27:54,210 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:27:54,210 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:27:54,210 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:27:54,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:27:54,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:27:54,217 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1F38B14D78964EA3A7D3FEB86E480FF7/B8DC469D6EE946709C7BBFFE1BA32E1D/.tez/application_1463524072159_0001
2016-05-17 22:27:54,219 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/STRICT/_temporary
2016-05-17 22:27:54,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:02.275
2016-05-17 22:27:54,301 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:54,301 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:54,378 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-17 22:27:54,414 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.077
2016-05-17 22:27:54,415 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:54,415 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:54,415 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:54,438 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:54,465 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/STRICT
2016-05-17 22:27:54,561 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:54,561 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:54,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/STRICT"]
2016-05-17 22:27:54,563 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:54,563 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:54,563 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:54,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...emoval/value/VALID/STRICT
2016-05-17 22:27:54,565 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:54,588 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:54,593 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:54,594 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:54,594 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:54,605 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B/.tez/application_1463524074593_0001 doesn't exist and is created
2016-05-17 22:27:54,632 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:54,632 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:54,633 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B/.tez/application_1463524074593_0001
2016-05-17 22:27:54,638 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524074593_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:54,659 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:54,660 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:54,662 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:54,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:54,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B/.tez/application_1463524074593_0001/localmode-log-dir
2016-05-17 22:27:54,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B/.tez/application_1463524074593_0001/localmode-log-dir/history.txt.appattempt_1463524074593_0001_000000, maxErrors=10
2016-05-17 22:27:54,663 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:54,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524074593_0001_000000, appSubmitTime=1463524074638, launchTime=1463524074638
2016-05-17 22:27:54,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:54,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:54,673 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:54,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:54,680 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41789
2016-05-17 22:27:54,683 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:54,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:54,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524074593_0001_000000, startTime=1463524074685
2016-05-17 22:27:54,686 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:54,733 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:54,733 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:54,733 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emoval/value/VALID/STRICT, applicationId=application_1463524074593_0001, dagName=[5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B] test/(1/1) ...emoval/value/VALID/STRICT
2016-05-17 22:27:54,769 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B] test/(1/1) ...emoval/value/VALID/STRICT
2016-05-17 22:27:54,775 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524074593_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B/.tez/application_1463524074593_0001/localmode-log-dir/dag_1463524074593_0001_1.dot
2016-05-17 22:27:54,777 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B/.tez/application_1463524074593_0001/localmode-log-dir/dag_1463524074593_0001_1-tez-dag.pb.txt
2016-05-17 22:27:54,840 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524074593_0001_1
2016-05-17 22:27:54,840 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:54,840 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B] test/(1/1) ...emoval/value/VALID/STRICT
2016-05-17 22:27:54 Running Dag: dag_1463524074593_0001_1
2016-05-17 22:27:54,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524074593_0001_1, submitTime=1463524074769
2016-05-17 22:27:54,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:27:54,860 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BA97D3AC75864C8796E1429BB4768C2E
2016-05-17 22:27:54,860 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:27:54,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:27:54,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524074593_0001_1, initTime=1463524074858
2016-05-17 22:27:54,862 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524074593_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:27:54,863 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:27:54,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:DAG_STARTED]: dagID=dag_1463524074593_0001_1, startTime=1463524074862
2016-05-17 22:27:54,863 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:27:54,863 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524074593_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:27:54,864 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EEAA62A3A3C04F14B801FB534F5B258D : {438C47E6A8A84380A59A05894C0EE404={InputName=438C47E6A8A84380A59A05894C0EE404}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:27:54,864 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 438C47E6A8A84380A59A05894C0EE404, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:27:54,864 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emoval/value/VALID/STRICT, applicationId=application_1463524074593_0001, dagName=[5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B] test/(1/1) ...emoval/value/VALID/STRICT
2016-05-17 22:27:54,864 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:54,865 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:54,865 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:54,865 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]. Starting root input initializers: 1
2016-05-17 22:27:54,866 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:27:54,866 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:54,866 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 438C47E6A8A84380A59A05894C0EE404 on vertex vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:54,875 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:27:54,876 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:54,876 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:54,877 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:54,877 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CF520062438D4BE4AB6659C771B31F22, vertexId=vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:27:54,877 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CF520062438D4BE4AB6659C771B31F22, vertexId=vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:54,888 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:27:54,888 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 438C47E6A8A84380A59A05894C0EE404 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:27:54,889 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CF520062438D4BE4AB6659C771B31F22, vertexName=BA97D3AC75864C8796E1429BB4768C2E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:27:54,890 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CF520062438D4BE4AB6659C771B31F22, vertexId=vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:54,890 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524074593_0001, with dag id: dag_1463524074593_0001_1
2016-05-17 22:27:54,889 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:27:54,891 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:27:54,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BA97D3AC75864C8796E1429BB4768C2E, vertexId=vertex_1463524074593_0001_1_01, initRequestedTime=1463524074866, initedTime=1463524074894, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:27:54,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:54,894 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:27:54,896 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:27:54,899 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 438C47E6A8A84380A59A05894C0EE404 on vertex vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:54,899 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:54,900 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {438C47E6A8A84380A59A05894C0EE404=forAllWorkUnits=true, update=[1]}
2016-05-17 22:27:54,901 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] parallelism set to 1
2016-05-17 22:27:54,902 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:54,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524074593_0001_1_00, eventCount=1
2016-05-17 22:27:54,906 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:54,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EEAA62A3A3C04F14B801FB534F5B258D, vertexId=vertex_1463524074593_0001_1_00, initRequestedTime=1463524074864, initedTime=1463524074906, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:27:54,907 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] with distanceFromRoot: 0
2016-05-17 22:27:54,907 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:27:54,909 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:54,909 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:54,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:54,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524074593_0001_1_00, startRequestedTime=1463524074909, startedTime=1463524074909
2016-05-17 22:27:54,911 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:54,913 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:54,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:TASK_STARTED]: vertexName=EEAA62A3A3C04F14B801FB534F5B258D, taskId=task_1463524074593_0001_1_00_000000, scheduledTime=1463524074913, launchTime=1463524074913
2016-05-17 22:27:54,914 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524074593_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:54,915 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524074593_0001_1_00 for vertex: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E] numStartedSources: 1 numSources: 1
2016-05-17 22:27:54,915 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E] with distanceFromRoot: 1
2016-05-17 22:27:54,915 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:54,916 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EEAA62A3A3C04F14B801FB534F5B258D in vertex: BA97D3AC75864C8796E1429BB4768C2E
2016-05-17 22:27:54,917 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BA97D3AC75864C8796E1429BB4768C2E with 1 source tasks and 1 pending tasks
2016-05-17 22:27:54,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:54,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524074593_0001_1_01, startRequestedTime=1463524074916, startedTime=1463524074916
2016-05-17 22:27:54,918 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:54,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B] test/(1/1) ...emoval/value/VALID/STRICT, VertexName: EEAA62A3A3C04F14B801FB534F5B258D, VertexParallelism: 1, TaskAttemptID:attempt_1463524074593_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=438C47E6A8A84380A59A05894C0EE404, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA97D3AC75864C8796E1429BB4768C2E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:54,922 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524074593_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:54,922 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:54,924 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:27:54,925 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:27:54,925 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:27:54,925 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524074593_0001_00_000001
2016-05-17 22:27:54,925 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:54,930 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524074593_0001_00_000001
2016-05-17 22:27:54,930 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:54,931 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:54,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524074593_0001_00_000001, launchTime=1463524074931
2016-05-17 22:27:54,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:54,932 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524074593_0001_00_000001
2016-05-17 22:27:54,933 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524074593_0001_1_00_000000_0] to container: [container_1463524074593_0001_00_000001]
2016-05-17 22:27:54,934 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:54,934 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524074593_0001_00_000001 given task: attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:54,935 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524074593_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:54,935 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524074593_0001_1_00_000000_0] started. Is using containerId: [container_1463524074593_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:27:54,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:54,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EEAA62A3A3C04F14B801FB534F5B258D, taskAttemptId=attempt_1463524074593_0001_1_00_000000_0, startTime=1463524074935, containerId=container_1463524074593_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524074593_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:27:54 Starting to run new task attempt: attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:54,937 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524074593_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:54,937 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:54,938 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:54,938 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524074593_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:54,938 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:54,938 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:54,939 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:54,939 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:54,939 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:54,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B] test/(1/1) ...emoval/value/VALID/STRICT, VertexName: EEAA62A3A3C04F14B801FB534F5B258D, VertexParallelism: 1, TaskAttemptID:attempt_1463524074593_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=438C47E6A8A84380A59A05894C0EE404, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA97D3AC75864C8796E1429BB4768C2E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:54,939 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:54,940 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:54,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:54,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:54,942 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524074593_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:27:54,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=438C47E6A8A84380A59A05894C0EE404, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:27:54,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:54,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:54,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 438C47E6A8A84380A59A05894C0EE404
2016-05-17 22:27:54,944 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 438C47E6A8A84380A59A05894C0EE404
2016-05-17 22:27:54,949 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:54,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BA97D3AC75864C8796E1429BB4768C2E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:27:54,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:54,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BA97D3AC75864C8796E1429BB4768C2E
2016-05-17 22:27:54,956 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:27:54,956 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 438C47E6A8A84380A59A05894C0EE404
2016-05-17 22:27:54,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 438C47E6A8A84380A59A05894C0EE404
2016-05-17 22:27:54,960 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:27:54,960 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BA97D3AC75864C8796E1429BB4768C2E
2016-05-17 22:27:54,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BA97D3AC75864C8796E1429BB4768C2E
2016-05-17 22:27:54,968 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EEAA62A3A3C04F14B801FB534F5B258D, ordinal: 0
2016-05-17 22:27:54,968 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEAA62A3A3C04F14B801FB534F5B258D, mem on start (mb), free: 143, total: 281, max: 672
2016-05-17 22:27:54,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:54,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:54,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:54,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:54,969 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:54,970 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:27:54,970 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:54,970 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:27:54,971 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:54,972 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:27:54,972 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 438C47E6A8A84380A59A05894C0EE404, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:27:54,972 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BA97D3AC75864C8796E1429BB4768C2E, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:27:54,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:54,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 438C47E6A8A84380A59A05894C0EE404 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:54,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 438C47E6A8A84380A59A05894C0EE404
2016-05-17 22:27:54,974 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:54,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 438C47E6A8A84380A59A05894C0EE404
2016-05-17 22:27:54,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:54,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:54,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:54,975 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:54,976 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:27:54,977 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5c21906f
2016-05-17 22:27:54,979 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5c21906f
2016-05-17 22:27:54,980 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:27:54,980 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:27:54,983 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:54,984 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 438C47E6A8A84380A59A05894C0EE404
2016-05-17 22:27:54,984 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: 17B70DA2E3E741D18FA216C43D655725
2016-05-17 22:27:54,984 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] 17B70DA2E3E741D18FA216C43D655725
2016-05-17 22:27:54,987 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3a33de17; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@2339d5db; comparator=cascading.tuple.hadoop.util.TupleComparator@2a072eee; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:27:54,987 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:27:54,988 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:27:54,988 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:55,109 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:27:55,110 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:27:55,110 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:27:55,110 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:27:55,110 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:27:55,111 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:55,111 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:55,111 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EEAA62A3A3C04F14B801FB534F5B258D, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:55,115 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:55,115 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEAA62A3A3C04F14B801FB534F5B258D, mem on close (mb), free: 141, total: 382, max: 672
2016-05-17 22:27:55,116 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:27:55,116 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:27:55,116 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:27:55,117 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:27:55,121 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:27:55 Completed running task attempt: attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:55,126 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524074593_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:55,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524074593_0001_1_00, eventCount=1
2016-05-17 22:27:55,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:55,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524075112, Process_Duration=3, Process_End_Time=1463524075115, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:27:55,130 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:55,131 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:55,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EEAA62A3A3C04F14B801FB534F5B258D, taskAttemptId=attempt_1463524074593_0001_1_00_000000_0, startTime=1463524074935, finishTime=1463524075128, timeTaken=193, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075112, Process_Duration=3, Process_End_Time=1463524075115, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:55,131 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524074593_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:55,132 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:55,133 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524074593_0001_1_00_000000_0
2016-05-17 22:27:55,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:55,131 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524074593_0001_00_000001
2016-05-17 22:27:55,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:TASK_FINISHED]: vertexName=EEAA62A3A3C04F14B801FB534F5B258D, taskId=task_1463524074593_0001_1_00_000000, startTime=1463524074935, finishTime=1463524075133, timeTaken=198, status=SUCCEEDED, successfulAttemptID=attempt_1463524074593_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075112, Process_Duration=3, Process_End_Time=1463524075115, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:55,135 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524074593_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:55,135 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] : 1
2016-05-17 22:27:55,135 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:55,136 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:55,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:VERTEX_FINISHED]: vertexName=EEAA62A3A3C04F14B801FB534F5B258D, vertexId=vertex_1463524074593_0001_1_00, initRequestedTime=1463524074864, initedTime=1463524074906, startRequestedTime=1463524074909, startedTime=1463524074909, finishTime=1463524075135, timeTaken=226, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075112, Process_Duration=3, Process_End_Time=1463524075115, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524074935, firstTasksToStart=[ task_1463524074593_0001_1_00_000000 ], lastTaskFinishTime=1463524075128, lastTasksToFinish=[ task_1463524074593_0001_1_00_000000 ], minTaskDuration=193, maxTaskDuration=193, avgTaskDuration=193.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524074593_0001_1_00_000000 ], longestDurationTasks=[ task_1463524074593_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:55,138 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:55,138 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:55,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:55,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524074593_0001_00_000001, stoppedTime=1463524075139, exitStatus=0
2016-05-17 22:27:55,140 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:55,140 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E] attempt: attempt_1463524074593_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:27:55,141 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BA97D3AC75864C8796E1429BB4768C2E
2016-05-17 22:27:55,141 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:55,141 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:55,142 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524074593_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:27:55,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:55,139 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524074593_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:27:55,142 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524074593_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:55,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:TASK_STARTED]: vertexName=BA97D3AC75864C8796E1429BB4768C2E, taskId=task_1463524074593_0001_1_01_000000, scheduledTime=1463524075142, launchTime=1463524075142
2016-05-17 22:27:55,143 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524074593_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:55,143 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:55,143 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:27:55,143 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B] test/(1/1) ...emoval/value/VALID/STRICT, VertexName: BA97D3AC75864C8796E1429BB4768C2E, VertexParallelism: 1, TaskAttemptID:attempt_1463524074593_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EEAA62A3A3C04F14B801FB534F5B258D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CF520062438D4BE4AB6659C771B31F22, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:55,144 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524074593_0001_00_000001, Exiting
2016-05-17 22:27:55,144 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524074593_0001_00_000001
2016-05-17 22:27:55,144 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524074593_0001_00_000001
2016-05-17 22:27:55,145 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524074593_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:55,146 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:55,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524074593_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:55,146 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524074593_0001_00_000001 completed successfully
2016-05-17 22:27:55,148 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524074593_0001_00_000002
2016-05-17 22:27:55,148 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:55,150 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:27:55,151 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524074593_0001_00_000002
2016-05-17 22:27:55,152 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:27:55,154 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:55,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:55,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524074593_0001_00_000002, launchTime=1463524075154
2016-05-17 22:27:55,155 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524074593_0001_00_000002
2016-05-17 22:27:55,156 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524074593_0001_1_01_000000_0] to container: [container_1463524074593_0001_00_000002]
2016-05-17 22:27:55,156 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:55,156 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524074593_0001_00_000002 given task: attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,156 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524074593_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,157 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524074593_0001_1_01_000000_0] started. Is using containerId: [container_1463524074593_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:27:55,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BA97D3AC75864C8796E1429BB4768C2E, taskAttemptId=attempt_1463524074593_0001_1_01_000000_0, startTime=1463524075156, containerId=container_1463524074593_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524074593_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:27:55,157 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524074593_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:55 Starting to run new task attempt: attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,157 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,157 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524074593_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:55,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:55,158 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:55,158 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:55,158 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:55,160 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:55,160 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:55,160 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:55,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B] test/(1/1) ...emoval/value/VALID/STRICT, VertexName: BA97D3AC75864C8796E1429BB4768C2E, VertexParallelism: 1, TaskAttemptID:attempt_1463524074593_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EEAA62A3A3C04F14B801FB534F5B258D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CF520062438D4BE4AB6659C771B31F22, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:55,160 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:55,162 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,162 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:55,162 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,163 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524074593_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:27:55,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,163 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EEAA62A3A3C04F14B801FB534F5B258D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:27:55,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:55,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:55,169 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:55,171 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:27:55,171 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:27:55,173 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:55,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CF520062438D4BE4AB6659C771B31F22, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:27:55,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:55,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CF520062438D4BE4AB6659C771B31F22
2016-05-17 22:27:55,175 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:27:55,175 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CF520062438D4BE4AB6659C771B31F22
2016-05-17 22:27:55,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:55,192 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BA97D3AC75864C8796E1429BB4768C2E, ordinal: 1
2016-05-17 22:27:55,192 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA97D3AC75864C8796E1429BB4768C2E, mem on start (mb), free: 133, total: 382, max: 672
2016-05-17 22:27:55,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:55,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:55,199 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:27:55,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CF520062438D4BE4AB6659C771B31F22
2016-05-17 22:27:55,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:55,200 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:55,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:27:55,200 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:27:55,201 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:55,201 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:55,201 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:27:55,202 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EEAA62A3A3C04F14B801FB534F5B258D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:27:55,202 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CF520062438D4BE4AB6659C771B31F22, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:27:55,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:55,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:55,205 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:55,205 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:27:55,206 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EEAA62A3A3C04F14B801FB534F5B258D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:27:55,206 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:27:55,206 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:27:55,206 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:27:55,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:55,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EEAA62A3A3C04F14B801FB534F5B258D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:55,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:55,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:55,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:55,212 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,213 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524074593_0001_1_00_000000_0_10009, runDuration: 171557, ]
2016-05-17 22:27:55,215 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524074593_0001_1_00_000000_0_10009] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524075215, TimeTaken=1, Rate=0.62 MB/s
2016-05-17 22:27:55,216 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:55,216 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:27:55,216 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [EEAA62A3A3C04F14B801FB534F5B258D] #2 in 2ms
2016-05-17 22:27:55,216 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EEAA62A3A3C04F14B801FB534F5B258D] #2, status:false, isInterrupted:false
2016-05-17 22:27:55,217 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:27:55,218 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: 17B70DA2E3E741D18FA216C43D655725
2016-05-17 22:27:55,218 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:27:55,219 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/STRICT"], id: CF520062438D4BE4AB6659C771B31F22
2016-05-17 22:27:55,219 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:27:55,219 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:55,220 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:27:55,220 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/STRICT"]
2016-05-17 22:27:55,221 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:27:55,221 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] 17B70DA2E3E741D18FA216C43D655725, for 1 inputs
2016-05-17 22:27:55,221 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:55,222 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:27:55,222 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BA97D3AC75864C8796E1429BB4768C2E, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:55,225 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@b1aacdb; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@44a638f
2016-05-17 22:27:55,227 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:27:55,228 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:27:55,229 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,229 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524074593_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:27:55,231 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,231 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA97D3AC75864C8796E1429BB4768C2E, mem on close (mb), free: 131, total: 382, max: 672
2016-05-17 22:27:55,231 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EEAA62A3A3C04F14B801FB534F5B258D
2016-05-17 22:27:55,232 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:27:55,233 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:27:55,233 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:55,233 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:27:55,233 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:27:55 Completed running task attempt: attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,235 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524074593_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:55,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524075222, Process_Duration=9, Process_End_Time=1463524075231, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=1]]
2016-05-17 22:27:55,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:55,238 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:55,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BA97D3AC75864C8796E1429BB4768C2E, taskAttemptId=attempt_1463524074593_0001_1_01_000000_0, startTime=1463524075156, finishTime=1463524075236, timeTaken=80, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075222, Process_Duration=9, Process_End_Time=1463524075231, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:55,239 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:55,239 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524074593_0001_00_000002
2016-05-17 22:27:55,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524074593_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:55,240 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:55,240 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524074593_0001_1_01_000000_0
2016-05-17 22:27:55,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:TASK_FINISHED]: vertexName=BA97D3AC75864C8796E1429BB4768C2E, taskId=task_1463524074593_0001_1_01_000000, startTime=1463524075156, finishTime=1463524075240, timeTaken=84, status=SUCCEEDED, successfulAttemptID=attempt_1463524074593_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075222, Process_Duration=9, Process_End_Time=1463524075231, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:55,241 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524074593_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:55,241 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E] : 1
2016-05-17 22:27:55,242 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:55,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:VERTEX_FINISHED]: vertexName=BA97D3AC75864C8796E1429BB4768C2E, vertexId=vertex_1463524074593_0001_1_01, initRequestedTime=1463524074866, initedTime=1463524074894, startRequestedTime=1463524074916, startedTime=1463524074916, finishTime=1463524075242, timeTaken=326, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075222, Process_Duration=9, Process_End_Time=1463524075231, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1, vertexStats=firstTaskStartTime=1463524075156, firstTasksToStart=[ task_1463524074593_0001_1_01_000000 ], lastTaskFinishTime=1463524075236, lastTasksToFinish=[ task_1463524074593_0001_1_01_000000 ], minTaskDuration=80, maxTaskDuration=80, avgTaskDuration=80.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524074593_0001_1_01_000000 ], longestDurationTasks=[ task_1463524074593_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:55,243 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:55,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:55,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524074593_0001_00_000002, stoppedTime=1463524075243, exitStatus=0
2016-05-17 22:27:55,244 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:55,244 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:55,244 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524074593_0001_1
2016-05-17 22:27:55,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524074593_0001_1
2016-05-17 22:27:55,246 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CF520062438D4BE4AB6659C771B31F22 for vertex: vertex_1463524074593_0001_1_01 [BA97D3AC75864C8796E1429BB4768C2E]
2016-05-17 22:27:55,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:55,247 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524074593_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 22:27:55,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:55,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:55,256 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524074593_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:55,259 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524074593_0001_00_000002, Exiting
2016-05-17 22:27:55,259 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524074593_0001_00_000002
2016-05-17 22:27:55,259 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524074593_0001_00_000002
2016-05-17 22:27:55,261 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524074593_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:55,261 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524074593_0001_00_000002 completed successfully
2016-05-17 22:27:55,264 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463524074593_0001_1_00 [EEAA62A3A3C04F14B801FB534F5B258D]
2016-05-17 22:27:55,266 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524074593_0001_1, queueSize=0
2016-05-17 22:27:55,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:27:55,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524074593_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524074593_0001_1, startTime=1463524074862, finishTime=1463524075264, timeTaken=402, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2891, FILE_BYTES_WRITTEN=704, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=14, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=801112064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=8, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927048150334, Process_Duration=12, Process_End_Time=2927048150346, Read_Duration=2, Tuples_Read=17, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=1
2016-05-17 22:27:55,268 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524074593_0001_1 finished with state: SUCCEEDED
2016-05-17 22:27:55,268 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524074593_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:27:55,268 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524074593_0001_00_000002
2016-05-17 22:27:55,268 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524074593_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:27:55 Completed Dag: dag_1463524074593_0001_1
2016-05-17 22:27:55,270 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524074593_0001_1, dagState=SUCCEEDED
2016-05-17 22:27:55,270 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524074593_0001_1_post
2016-05-17 22:27:55,271 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:55,273 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:27:55,277 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emoval/value/VALID/STRICT, applicationId=application_1463524074593_0001
2016-05-17 22:27:55,282 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:27:55,282 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:27:55,282 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:27:55,282 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:27:55,282 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:27:55,283 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:27:55,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:27:55,283 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:27:55,284 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:27:55,284 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:27:55,284 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:27:55,285 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:27:55,285 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:27:55,292 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/5F9E2031BAD241C190B44B5340919E50/FB26F84FBAA4493C8F291AB6E2C4E44B/.tez/application_1463524074593_0001
2016-05-17 22:27:55,293 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/STRICT/_temporary
2016-05-17 22:27:55,293 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.731
2016-05-17 22:27:55,365 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:55,366 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:55,445 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.042
2016-05-17 22:27:55,469 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.066
2016-05-17 22:27:55,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:55,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:55,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:55,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:55,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/STRICT/STRICT
2016-05-17 22:27:55,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:55,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:55,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/STRICT/STRICT"]
2016-05-17 22:27:55,571 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:55,571 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:55,571 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:55,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...moval/value/STRICT/STRICT
2016-05-17 22:27:55,572 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:55,587 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:55,591 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:55,592 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:55,592 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:55,602 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2/.tez/application_1463524075592_0001 doesn't exist and is created
2016-05-17 22:27:55,626 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:55,626 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:55,627 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2/.tez/application_1463524075592_0001
2016-05-17 22:27:55,632 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524075592_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:55,645 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:55,646 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:55,648 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:55,648 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:55,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2/.tez/application_1463524075592_0001/localmode-log-dir
2016-05-17 22:27:55,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2/.tez/application_1463524075592_0001/localmode-log-dir/history.txt.appattempt_1463524075592_0001_000000, maxErrors=10
2016-05-17 22:27:55,650 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:55,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524075592_0001_000000, appSubmitTime=1463524075631, launchTime=1463524075631
2016-05-17 22:27:55,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:55,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:55,662 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:55,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:55,664 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53951
2016-05-17 22:27:55,666 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:55,669 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524075592_0001_000000, startTime=1463524075669
2016-05-17 22:27:55,669 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:55,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:55,726 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:55,726 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:55,726 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...moval/value/STRICT/STRICT, applicationId=application_1463524075592_0001, dagName=[CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2] test/(1/1) ...moval/value/STRICT/STRICT
2016-05-17 22:27:55,745 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2] test/(1/1) ...moval/value/STRICT/STRICT
2016-05-17 22:27:55,747 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524075592_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2/.tez/application_1463524075592_0001/localmode-log-dir/dag_1463524075592_0001_1.dot
2016-05-17 22:27:55,748 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2/.tez/application_1463524075592_0001/localmode-log-dir/dag_1463524075592_0001_1-tez-dag.pb.txt
2016-05-17 22:27:55,782 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524075592_0001_1
2016-05-17 22:27:55,782 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:55,782 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2] test/(1/1) ...moval/value/STRICT/STRICT
2016-05-17 22:27:55 Running Dag: dag_1463524075592_0001_1
2016-05-17 22:27:55,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524075592_0001_1, submitTime=1463524075746
2016-05-17 22:27:55,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:27:55,800 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B396C39E52564BDCBE328718C48B6100
2016-05-17 22:27:55,801 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:27:55,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524075592_0001_1, initTime=1463524075800
2016-05-17 22:27:55,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:27:55,802 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524075592_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:27:55,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:DAG_STARTED]: dagID=dag_1463524075592_0001_1, startTime=1463524075802
2016-05-17 22:27:55,803 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:27:55,803 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...moval/value/STRICT/STRICT, applicationId=application_1463524075592_0001, dagName=[CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2] test/(1/1) ...moval/value/STRICT/STRICT
2016-05-17 22:27:55,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:27:55,803 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524075592_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:27:55,804 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EBDBD67D9896457186B9A100C69CD1BD : {6E0BCF836EEE4AE6B362BEF08B9664BD={InputName=6E0BCF836EEE4AE6B362BEF08B9664BD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:27:55,804 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6E0BCF836EEE4AE6B362BEF08B9664BD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:27:55,804 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:55,804 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:55,804 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:55,805 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]. Starting root input initializers: 1
2016-05-17 22:27:55,806 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:27:55,806 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:55,807 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6E0BCF836EEE4AE6B362BEF08B9664BD on vertex vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:55,810 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:27:55,811 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:27:55,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:55,811 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6E0BCF836EEE4AE6B362BEF08B9664BD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:27:55,812 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:55,812 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:55,812 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CDCB6D5802AB43C5B4CCC4C54E52BD02, vertexId=vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:27:55,813 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CDCB6D5802AB43C5B4CCC4C54E52BD02, vertexId=vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:55,816 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:27:55,816 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:27:55,816 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CDCB6D5802AB43C5B4CCC4C54E52BD02, vertexName=B396C39E52564BDCBE328718C48B6100, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:27:55,822 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CDCB6D5802AB43C5B4CCC4C54E52BD02, vertexId=vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:55,822 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:27:55,823 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6E0BCF836EEE4AE6B362BEF08B9664BD on vertex vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:55,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:55,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B396C39E52564BDCBE328718C48B6100, vertexId=vertex_1463524075592_0001_1_01, initRequestedTime=1463524075806, initedTime=1463524075824, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:27:55,825 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:27:55,825 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:55,826 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6E0BCF836EEE4AE6B362BEF08B9664BD=forAllWorkUnits=true, update=[1]}
2016-05-17 22:27:55,826 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] parallelism set to 1
2016-05-17 22:27:55,826 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:55,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524075592_0001_1_00, eventCount=1
2016-05-17 22:27:55,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:55,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EBDBD67D9896457186B9A100C69CD1BD, vertexId=vertex_1463524075592_0001_1_00, initRequestedTime=1463524075804, initedTime=1463524075828, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:27:55,828 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] with distanceFromRoot: 0
2016-05-17 22:27:55,828 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:27:55,828 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:55,829 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:55,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524075592_0001_1_00, startRequestedTime=1463524075828, startedTime=1463524075828
2016-05-17 22:27:55,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:55,829 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:55,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:55,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:TASK_STARTED]: vertexName=EBDBD67D9896457186B9A100C69CD1BD, taskId=task_1463524075592_0001_1_00_000000, scheduledTime=1463524075830, launchTime=1463524075830
2016-05-17 22:27:55,830 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524075592_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:55,831 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524075592_0001_1_00 for vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] numStartedSources: 1 numSources: 1
2016-05-17 22:27:55,831 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] with distanceFromRoot: 1
2016-05-17 22:27:55,831 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:55,832 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EBDBD67D9896457186B9A100C69CD1BD in vertex: B396C39E52564BDCBE328718C48B6100
2016-05-17 22:27:55,832 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B396C39E52564BDCBE328718C48B6100 with 1 source tasks and 1 pending tasks
2016-05-17 22:27:55,832 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:55,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524075592_0001_1_01, startRequestedTime=1463524075831, startedTime=1463524075831
2016-05-17 22:27:55,832 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:55,833 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2] test/(1/1) ...moval/value/STRICT/STRICT, VertexName: EBDBD67D9896457186B9A100C69CD1BD, VertexParallelism: 1, TaskAttemptID:attempt_1463524075592_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6E0BCF836EEE4AE6B362BEF08B9664BD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B396C39E52564BDCBE328718C48B6100, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:55,833 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524075592_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:55,833 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:55,834 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:27:55,835 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:27:55,835 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:27:55,836 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524075592_0001_00_000001
2016-05-17 22:27:55,836 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:55,837 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524075592_0001, with dag id: dag_1463524075592_0001_1
2016-05-17 22:27:55,839 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524075592_0001_00_000001
2016-05-17 22:27:55,840 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:55,845 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:55,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:55,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524075592_0001_00_000001, launchTime=1463524075845
2016-05-17 22:27:55,847 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524075592_0001_00_000001
2016-05-17 22:27:55,848 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524075592_0001_1_00_000000_0] to container: [container_1463524075592_0001_00_000001]
2016-05-17 22:27:55,848 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:55,849 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524075592_0001_00_000001 given task: attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,849 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524075592_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,849 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524075592_0001_1_00_000000_0] started. Is using containerId: [container_1463524075592_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:27:55,850 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:55,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EBDBD67D9896457186B9A100C69CD1BD, taskAttemptId=attempt_1463524075592_0001_1_00_000000_0, startTime=1463524075849, containerId=container_1463524075592_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524075592_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:27:55,851 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524075592_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:55 Starting to run new task attempt: attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,851 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,851 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:55,851 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:55,851 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:55,851 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:55,851 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:55,851 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524075592_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:55,855 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:55,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2] test/(1/1) ...moval/value/STRICT/STRICT, VertexName: EBDBD67D9896457186B9A100C69CD1BD, VertexParallelism: 1, TaskAttemptID:attempt_1463524075592_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6E0BCF836EEE4AE6B362BEF08B9664BD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B396C39E52564BDCBE328718C48B6100, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:55,855 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:55,855 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:55,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B396C39E52564BDCBE328718C48B6100, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:27:55,858 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524075592_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:27:55,858 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:55,858 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B396C39E52564BDCBE328718C48B6100
2016-05-17 22:27:55,860 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:27:55,860 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B396C39E52564BDCBE328718C48B6100
2016-05-17 22:27:55,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B396C39E52564BDCBE328718C48B6100
2016-05-17 22:27:55,860 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6E0BCF836EEE4AE6B362BEF08B9664BD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:27:55,860 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:55,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:55,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6E0BCF836EEE4AE6B362BEF08B9664BD
2016-05-17 22:27:55,861 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6E0BCF836EEE4AE6B362BEF08B9664BD
2016-05-17 22:27:55,893 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:27:55,893 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6E0BCF836EEE4AE6B362BEF08B9664BD
2016-05-17 22:27:55,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6E0BCF836EEE4AE6B362BEF08B9664BD
2016-05-17 22:27:55,899 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EBDBD67D9896457186B9A100C69CD1BD, ordinal: 0
2016-05-17 22:27:55,899 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EBDBD67D9896457186B9A100C69CD1BD, mem on start (mb), free: 265, total: 491, max: 672
2016-05-17 22:27:55,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:55,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:55,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:55,899 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:55,899 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:27:55,899 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:27:55,900 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:55,900 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:55,900 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:27:55,900 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B396C39E52564BDCBE328718C48B6100, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:27:55,900 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6E0BCF836EEE4AE6B362BEF08B9664BD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:27:55,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:55,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6E0BCF836EEE4AE6B362BEF08B9664BD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:55,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:55,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:55,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6E0BCF836EEE4AE6B362BEF08B9664BD
2016-05-17 22:27:55,901 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:55,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6E0BCF836EEE4AE6B362BEF08B9664BD
2016-05-17 22:27:55,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:55,902 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,902 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:27:55,904 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6e58d51c
2016-05-17 22:27:55,908 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:55,909 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6e58d51c
2016-05-17 22:27:55,909 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:27:55,909 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:27:55,909 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 6E0BCF836EEE4AE6B362BEF08B9664BD
2016-05-17 22:27:55,910 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: CCBBE52FFD044A898A445F9CCF0E108E
2016-05-17 22:27:55,910 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] CCBBE52FFD044A898A445F9CCF0E108E
2016-05-17 22:27:55,912 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1a00a506; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@20523a40; comparator=cascading.tuple.hadoop.util.TupleComparator@766e5053; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:27:55,912 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:27:55,913 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:27:55,913 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:55,926 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:27:55,927 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:27:55,927 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:27:55,927 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:27:55,927 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:27:55,928 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:55,929 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:55,929 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EBDBD67D9896457186B9A100C69CD1BD, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:55,932 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,932 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EBDBD67D9896457186B9A100C69CD1BD, mem on close (mb), free: 163, total: 491, max: 672
2016-05-17 22:27:55,933 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:27:55,933 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:27:55,933 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:27:55,933 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:27:55,937 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:27:55 Completed running task attempt: attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,941 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524075592_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:55,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524075592_0001_1_00, eventCount=1
2016-05-17 22:27:55,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524075929, Process_Duration=3, Process_End_Time=1463524075932, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:27:55,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:55,943 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:55,943 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:55,943 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524075592_0001_00_000001
2016-05-17 22:27:55,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EBDBD67D9896457186B9A100C69CD1BD, taskAttemptId=attempt_1463524075592_0001_1_00_000000_0, startTime=1463524075849, finishTime=1463524075942, timeTaken=93, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075929, Process_Duration=3, Process_End_Time=1463524075932, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:55,944 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524075592_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:55,944 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:55,944 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524075592_0001_1_00_000000_0
2016-05-17 22:27:55,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:55,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:TASK_FINISHED]: vertexName=EBDBD67D9896457186B9A100C69CD1BD, taskId=task_1463524075592_0001_1_00_000000, startTime=1463524075849, finishTime=1463524075945, timeTaken=96, status=SUCCEEDED, successfulAttemptID=attempt_1463524075592_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075929, Process_Duration=3, Process_End_Time=1463524075932, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:55,945 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524075592_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:55,946 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] : 1
2016-05-17 22:27:55,946 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:55,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:VERTEX_FINISHED]: vertexName=EBDBD67D9896457186B9A100C69CD1BD, vertexId=vertex_1463524075592_0001_1_00, initRequestedTime=1463524075804, initedTime=1463524075828, startRequestedTime=1463524075828, startedTime=1463524075828, finishTime=1463524075946, timeTaken=118, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075929, Process_Duration=3, Process_End_Time=1463524075932, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524075849, firstTasksToStart=[ task_1463524075592_0001_1_00_000000 ], lastTaskFinishTime=1463524075942, lastTasksToFinish=[ task_1463524075592_0001_1_00_000000 ], minTaskDuration=93, maxTaskDuration=93, avgTaskDuration=93.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524075592_0001_1_00_000000 ], longestDurationTasks=[ task_1463524075592_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:55,947 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:55,947 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:55,948 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] attempt: attempt_1463524075592_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:27:55,948 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B396C39E52564BDCBE328718C48B6100
2016-05-17 22:27:55,948 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:55,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524075592_0001_00_000001, stoppedTime=1463524075948, exitStatus=0
2016-05-17 22:27:55,949 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:55,949 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:55,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:55,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:55,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:TASK_STARTED]: vertexName=B396C39E52564BDCBE328718C48B6100, taskId=task_1463524075592_0001_1_01_000000, scheduledTime=1463524075950, launchTime=1463524075950
2016-05-17 22:27:55,951 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:55,951 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524075592_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:55,951 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:55,951 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524075592_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:27:55,951 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2] test/(1/1) ...moval/value/STRICT/STRICT, VertexName: B396C39E52564BDCBE328718C48B6100, VertexParallelism: 1, TaskAttemptID:attempt_1463524075592_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EBDBD67D9896457186B9A100C69CD1BD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CDCB6D5802AB43C5B4CCC4C54E52BD02, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:55,952 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524075592_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:55,952 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:27:55,952 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:55,954 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524075592_0001_00_000002
2016-05-17 22:27:55,954 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:55,954 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524075592_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:27:55,954 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524075592_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:55,955 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524075592_0001_00_000001, Exiting
2016-05-17 22:27:55,955 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524075592_0001_00_000001
2016-05-17 22:27:55,955 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524075592_0001_00_000001
2016-05-17 22:27:55,956 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524075592_0001_00_000002
2016-05-17 22:27:55,957 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:27:55,958 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524075592_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:55,958 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524075592_0001_00_000001 completed successfully
2016-05-17 22:27:55,958 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:27:55,960 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:55,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:55,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524075592_0001_00_000002, launchTime=1463524075960
2016-05-17 22:27:55,961 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524075592_0001_00_000002
2016-05-17 22:27:55,962 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524075592_0001_1_01_000000_0] to container: [container_1463524075592_0001_00_000002]
2016-05-17 22:27:55,962 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:55,962 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524075592_0001_00_000002 given task: attempt_1463524075592_0001_1_01_000000_0
2016-05-17 22:27:55,962 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524075592_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524075592_0001_1_01_000000_0
2016-05-17 22:27:55,963 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524075592_0001_1_01_000000_0] started. Is using containerId: [container_1463524075592_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:27:55 Starting to run new task attempt: attempt_1463524075592_0001_1_01_000000_0
2016-05-17 22:27:55,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:55,963 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524075592_0001_1_01_000000_0
2016-05-17 22:27:55,963 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:55,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B396C39E52564BDCBE328718C48B6100, taskAttemptId=attempt_1463524075592_0001_1_01_000000_0, startTime=1463524075962, containerId=container_1463524075592_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524075592_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:27:55,963 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:55,964 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:55,963 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524075592_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:55,964 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:55,964 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524075592_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:55,964 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:55,965 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:55,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2] test/(1/1) ...moval/value/STRICT/STRICT, VertexName: B396C39E52564BDCBE328718C48B6100, VertexParallelism: 1, TaskAttemptID:attempt_1463524075592_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EBDBD67D9896457186B9A100C69CD1BD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CDCB6D5802AB43C5B4CCC4C54E52BD02, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:55,966 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:55,967 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524075592_0001_1_01_000000_0
2016-05-17 22:27:55,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:55,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,968 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524075592_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:27:55,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EBDBD67D9896457186B9A100C69CD1BD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:27:55,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CDCB6D5802AB43C5B4CCC4C54E52BD02, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:27:55,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:55,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:55,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:55,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CDCB6D5802AB43C5B4CCC4C54E52BD02
2016-05-17 22:27:55,971 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:27:55,971 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CDCB6D5802AB43C5B4CCC4C54E52BD02
2016-05-17 22:27:55,972 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:27:55,972 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:27:55,972 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:55,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:55,974 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:55,983 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:27:55,983 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CDCB6D5802AB43C5B4CCC4C54E52BD02
2016-05-17 22:27:55,991 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B396C39E52564BDCBE328718C48B6100, ordinal: 1
2016-05-17 22:27:55,991 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B396C39E52564BDCBE328718C48B6100, mem on start (mb), free: 155, total: 491, max: 672
2016-05-17 22:27:55,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:55,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:55,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:55,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:55,991 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:55,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:27:55,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:55,992 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:27:55,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:55,993 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:27:55,993 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CDCB6D5802AB43C5B4CCC4C54E52BD02, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:27:55,993 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EBDBD67D9896457186B9A100C69CD1BD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:27:55,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:55,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EBDBD67D9896457186B9A100C69CD1BD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:55,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:55,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:55,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:55,995 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:55,996 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:27:55,996 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EBDBD67D9896457186B9A100C69CD1BD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:27:55,997 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:27:55,997 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:27:55,997 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:27:55,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:55,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:55,999 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524075592_0001_1_01_000000_0
2016-05-17 22:27:56,000 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524075592_0001_1_00_000000_0_10014, runDuration: 80524, ]
2016-05-17 22:27:56,001 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524075592_0001_1_00_000000_0_10014] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524076001, TimeTaken=1, Rate=0.62 MB/s
2016-05-17 22:27:56,001 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:56,002 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:27:56,002 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [EBDBD67D9896457186B9A100C69CD1BD] #3 in 2ms
2016-05-17 22:27:56,002 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EBDBD67D9896457186B9A100C69CD1BD] #3, status:false, isInterrupted:false
2016-05-17 22:27:56,003 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:27:56,004 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:27:56,004 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:56,004 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:27:56,005 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:27:56,005 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:56,006 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:27:56,007 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:27:56,008 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: CCBBE52FFD044A898A445F9CCF0E108E
2016-05-17 22:27:56,008 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/STRICT/STRICT"], id: CDCB6D5802AB43C5B4CCC4C54E52BD02
2016-05-17 22:27:56,009 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/STRICT/STRICT"]
2016-05-17 22:27:56,009 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] CCBBE52FFD044A898A445F9CCF0E108E, for 1 inputs
2016-05-17 22:27:56,010 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B396C39E52564BDCBE328718C48B6100, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:56,011 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@3aa129ee; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2ac7e654
2016-05-17 22:27:56,014 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:56,016 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:56,017 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:56,018 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:27:56,019 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:27:56,020 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524075592_0001_1_01_000000_0
2016-05-17 22:27:56,022 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524075592_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:27:56,028 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524075592_0001_1_01_000000_0
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:56,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524076010, Process_Duration=18, Process_End_Time=1463524076028, Read_Duration=0, Tuples_Read=7]]
2016-05-17 22:27:56,036 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B396C39E52564BDCBE328718C48B6100, mem on close (mb), free: 151, total: 491, max: 672
2016-05-17 22:27:56,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B396C39E52564BDCBE328718C48B6100, index=1
2016-05-17 22:27:56,037 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:56,038 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:27:56,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:56,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B396C39E52564BDCBE328718C48B6100, taskAttemptId=attempt_1463524075592_0001_1_01_000000_0, startTime=1463524075962, finishTime=1463524076036, timeTaken=74, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524076010, Process_Duration=18, Process_End_Time=1463524076028, Read_Duration=0, Tuples_Read=7
2016-05-17 22:27:56,040 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524075592_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:27:56,040 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524075592_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:27:56,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:56,039 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:27:56,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:TASK_FINISHED]: vertexName=B396C39E52564BDCBE328718C48B6100, taskId=task_1463524075592_0001_1_01_000000, startTime=1463524075962, finishTime=1463524076041, timeTaken=79, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-17 22:27:56,043 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524075592_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:27:56,043 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] : 1
2016-05-17 22:27:56,043 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] because task failed: task_1463524075592_0001_1_01_000000
2016-05-17 22:27:56,043 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:27:56,044 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:27:56,044 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:56,044 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:56,046 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=CDCB6D5802AB43C5B4CCC4C54E52BD02, vertexId=vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:56,045 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:56,047 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:27:56,047 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:27:56,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B396C39E52564BDCBE328718C48B6100, sourceVertex=EBDBD67D9896457186B9A100C69CD1BD
2016-05-17 22:27:56,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B396C39E52564BDCBE328718C48B6100, sourceVertex=CDCB6D5802AB43C5B4CCC4C54E52BD02
2016-05-17 22:27:56,047 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524075592_0001_00_000002
2016-05-17 22:27:56,048 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524075592_0001_00_000002
2016-05-17 22:27:56,049 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524075592_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:27:56,049 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524075592_0001_00_000002 completed but with errors
2016-05-17 22:27:56,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:56,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:VERTEX_FINISHED]: vertexName=B396C39E52564BDCBE328718C48B6100, vertexId=vertex_1463524075592_0001_1_01, initRequestedTime=1463524075806, initedTime=1463524075824, startRequestedTime=1463524075831, startedTime=1463524075831, finishTime=1463524076057, timeTaken=226, status=FAILED, diagnostics=Task failed, taskId=task_1463524075592_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524075962, firstTasksToStart=[ task_1463524075592_0001_1_01_000000 ], lastTaskFinishTime=1463524076036, lastTasksToFinish=[ task_1463524075592_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:27:56,059 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:27:56,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:56,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524075592_0001_00_000002, stoppedTime=1463524076060, exitStatus=0
2016-05-17 22:27:56,062 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:56,063 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524075592_0001_1_01_000000_0 failed: true container: container_1463524075592_0001_00_000002 numFailedTAs: 0
2016-05-17 22:27:56,065 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524075592_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:27:56,065 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524075592_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:27:56,067 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 22:27:56,067 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:27:56,067 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524075592_0001_1
2016-05-17 22:27:56,067 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463524075592_0001_1_00 [EBDBD67D9896457186B9A100C69CD1BD]
2016-05-17 22:27:56,068 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: CDCB6D5802AB43C5B4CCC4C54E52BD02 for vertex: vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100]
2016-05-17 22:27:56,070 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524075592_0001_1, queueSize=0
2016-05-17 22:27:56,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:27:56,071 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524075592_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524075592_0001_1, startTime=1463524075802, finishTime=1463524076069, timeTaken=267, status=FAILED, diagnostics=Vertex failed, vertexName=B396C39E52564BDCBE328718C48B6100, vertexId=vertex_1463524075592_0001_1_01, diagnostics=[Task failed, taskId=task_1463524075592_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524075592_0001_1_01 [B396C39E52564BDCBE328718C48B6100] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524075929, Process_Duration=3, Process_End_Time=1463524075932, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:56,073 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524075592_0001_1 finished with state: FAILED
2016-05-17 22:27:56,073 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524075592_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:27:56,074 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524075592_0001_00_000002
2016-05-17 22:27:56,074 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524075592_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:27:56 Completed Dag: dag_1463524075592_0001_1
2016-05-17 22:27:56,075 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524075592_0001_1, dagState=FAILED
2016-05-17 22:27:56,075 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524075592_0001_1_post
2016-05-17 22:27:56,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:56,076 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:27:56,080 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...moval/value/STRICT/STRICT, applicationId=application_1463524075592_0001
2016-05-17 22:27:56,083 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:27:56,083 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:27:56,083 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:27:56,083 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:27:56,084 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:27:56,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:27:56,084 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:27:56,085 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:27:56,085 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:27:56,085 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:27:56,086 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:27:56,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:27:56,087 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:27:56,094 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/CAC9B3290C2848868DB4E221ECD3CEEC/C3C075AD1D6642128B5687140FD0DED2/.tez/application_1463524075592_0001
2016-05-17 22:27:56,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping all jobs
2016-05-17 22:27:56,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping: (1/1) ...moval/value/STRICT/STRICT
2016-05-17 22:27:56,095 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopped all jobs
2016-05-17 22:27:56,096 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/STRICT/STRICT/_temporary
2016-05-17 22:27:56,096 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.526
2016-05-17 22:27:56,126 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:56,126 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:56,185 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 22:27:56,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.056
2016-05-17 22:27:56,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:56,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:56,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:56,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:56,259 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/VALID
2016-05-17 22:27:56,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:56,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:56,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/VALID"]
2016-05-17 22:27:56,340 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:56,340 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:56,340 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:56,341 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...sRemoval/value/NONE/VALID
2016-05-17 22:27:56,341 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:56,360 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:56,365 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:56,365 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:56,366 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:56,375 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E/.tez/application_1463524076365_0001 doesn't exist and is created
2016-05-17 22:27:56,396 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:56,396 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:56,396 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E/.tez/application_1463524076365_0001
2016-05-17 22:27:56,401 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524076365_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:56,417 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:56,418 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:56,420 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:56,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:56,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E/.tez/application_1463524076365_0001/localmode-log-dir
2016-05-17 22:27:56,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E/.tez/application_1463524076365_0001/localmode-log-dir/history.txt.appattempt_1463524076365_0001_000000, maxErrors=10
2016-05-17 22:27:56,420 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:56,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524076365_0001_000000, appSubmitTime=1463524076400, launchTime=1463524076401
2016-05-17 22:27:56,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:56,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:56,430 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:56,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:56,434 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38887
2016-05-17 22:27:56,436 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:56,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524076365_0001_000000, startTime=1463524076438
2016-05-17 22:27:56,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:56,438 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:56,496 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:56,497 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:56,497 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sRemoval/value/NONE/VALID, applicationId=application_1463524076365_0001, dagName=[AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E] test/(1/1) ...sRemoval/value/NONE/VALID
2016-05-17 22:27:56,529 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E] test/(1/1) ...sRemoval/value/NONE/VALID
2016-05-17 22:27:56,531 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524076365_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E/.tez/application_1463524076365_0001/localmode-log-dir/dag_1463524076365_0001_1.dot
2016-05-17 22:27:56,533 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E/.tez/application_1463524076365_0001/localmode-log-dir/dag_1463524076365_0001_1-tez-dag.pb.txt
2016-05-17 22:27:56,567 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524076365_0001_1
2016-05-17 22:27:56,567 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:56,567 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E] test/(1/1) ...sRemoval/value/NONE/VALID
2016-05-17 22:27:56 Running Dag: dag_1463524076365_0001_1
2016-05-17 22:27:56,584 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524076365_0001_1, submitTime=1463524076529
2016-05-17 22:27:56,584 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:27:56,586 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FE85C7A2AB0C4CDA905D62FD2B11020A
2016-05-17 22:27:56,586 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:27:56,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:27:56,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524076365_0001_1, initTime=1463524076584
2016-05-17 22:27:56,587 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524076365_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:27:56,588 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:27:56,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:DAG_STARTED]: dagID=dag_1463524076365_0001_1, startTime=1463524076588
2016-05-17 22:27:56,588 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:27:56,589 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524076365_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:27:56,589 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B9DACBDE3BC249D6A6893046D4CF528E : {7F8F131755894368AEB28D20AAC001E4={InputName=7F8F131755894368AEB28D20AAC001E4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:27:56,589 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7F8F131755894368AEB28D20AAC001E4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:27:56,589 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,589 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,590 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,590 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]. Starting root input initializers: 1
2016-05-17 22:27:56,591 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:27:56,591 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,591 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sRemoval/value/NONE/VALID, applicationId=application_1463524076365_0001, dagName=[AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E] test/(1/1) ...sRemoval/value/NONE/VALID
2016-05-17 22:27:56,593 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7F8F131755894368AEB28D20AAC001E4 on vertex vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,596 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:27:56,596 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,596 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,596 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,597 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8407F60730E044A2A73BE13D5F43337D, vertexId=vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:27:56,597 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8407F60730E044A2A73BE13D5F43337D, vertexId=vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,599 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:27:56,600 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7F8F131755894368AEB28D20AAC001E4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:27:56,600 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8407F60730E044A2A73BE13D5F43337D, vertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:27:56,600 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8407F60730E044A2A73BE13D5F43337D, vertexId=vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,601 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:27:56,601 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:27:56,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:56,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, vertexId=vertex_1463524076365_0001_1_01, initRequestedTime=1463524076591, initedTime=1463524076603, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:27:56,604 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:27:56,604 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:27:56,605 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7F8F131755894368AEB28D20AAC001E4 on vertex vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,605 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:56,606 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7F8F131755894368AEB28D20AAC001E4=forAllWorkUnits=true, update=[1]}
2016-05-17 22:27:56,606 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] parallelism set to 1
2016-05-17 22:27:56,606 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:56,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524076365_0001_1_00, eventCount=1
2016-05-17 22:27:56,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B9DACBDE3BC249D6A6893046D4CF528E, vertexId=vertex_1463524076365_0001_1_00, initRequestedTime=1463524076589, initedTime=1463524076608, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:27:56,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:56,608 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] with distanceFromRoot: 0
2016-05-17 22:27:56,610 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:27:56,611 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,611 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,611 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:56,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524076365_0001_1_00, startRequestedTime=1463524076611, startedTime=1463524076611
2016-05-17 22:27:56,612 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:56,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:56,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:TASK_STARTED]: vertexName=B9DACBDE3BC249D6A6893046D4CF528E, taskId=task_1463524076365_0001_1_00_000000, scheduledTime=1463524076613, launchTime=1463524076613
2016-05-17 22:27:56,613 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524076365_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:56,614 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524076365_0001_1_00 for vertex: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A] numStartedSources: 1 numSources: 1
2016-05-17 22:27:56,614 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A] with distanceFromRoot: 1
2016-05-17 22:27:56,615 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,615 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B9DACBDE3BC249D6A6893046D4CF528E in vertex: FE85C7A2AB0C4CDA905D62FD2B11020A
2016-05-17 22:27:56,616 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FE85C7A2AB0C4CDA905D62FD2B11020A with 1 source tasks and 1 pending tasks
2016-05-17 22:27:56,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:56,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524076365_0001_1_01, startRequestedTime=1463524076615, startedTime=1463524076615
2016-05-17 22:27:56,617 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:56,618 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E] test/(1/1) ...sRemoval/value/NONE/VALID, VertexName: B9DACBDE3BC249D6A6893046D4CF528E, VertexParallelism: 1, TaskAttemptID:attempt_1463524076365_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7F8F131755894368AEB28D20AAC001E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:56,619 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524076365_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:56,619 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:56,621 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:27:56,621 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:27:56,621 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:27:56,622 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524076365_0001_00_000001
2016-05-17 22:27:56,622 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524076365_0001, with dag id: dag_1463524076365_0001_1
2016-05-17 22:27:56,622 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:56,626 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524076365_0001_00_000001
2016-05-17 22:27:56,626 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:56,627 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:56,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:56,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524076365_0001_00_000001, launchTime=1463524076627
2016-05-17 22:27:56,629 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524076365_0001_00_000001
2016-05-17 22:27:56,630 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524076365_0001_1_00_000000_0] to container: [container_1463524076365_0001_00_000001]
2016-05-17 22:27:56,630 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:56,631 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524076365_0001_00_000001 given task: attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,631 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524076365_0001_1_00_000000_0] started. Is using containerId: [container_1463524076365_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:27:56,631 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:56,631 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524076365_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B9DACBDE3BC249D6A6893046D4CF528E, taskAttemptId=attempt_1463524076365_0001_1_00_000000_0, startTime=1463524076631, containerId=container_1463524076365_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524076365_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:27:56,632 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524076365_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:56,632 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524076365_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:56 Starting to run new task attempt: attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,633 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,633 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:56,634 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:56,634 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:56,634 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:56,634 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:56,634 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:56,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E] test/(1/1) ...sRemoval/value/NONE/VALID, VertexName: B9DACBDE3BC249D6A6893046D4CF528E, VertexParallelism: 1, TaskAttemptID:attempt_1463524076365_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7F8F131755894368AEB28D20AAC001E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:56,635 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:56,635 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:56,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:56,636 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524076365_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:27:56,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7F8F131755894368AEB28D20AAC001E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:27:56,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:27:56,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:56,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:56,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FE85C7A2AB0C4CDA905D62FD2B11020A
2016-05-17 22:27:56,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:56,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7F8F131755894368AEB28D20AAC001E4
2016-05-17 22:27:56,640 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7F8F131755894368AEB28D20AAC001E4
2016-05-17 22:27:56,642 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:27:56,642 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FE85C7A2AB0C4CDA905D62FD2B11020A
2016-05-17 22:27:56,642 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:56,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FE85C7A2AB0C4CDA905D62FD2B11020A
2016-05-17 22:27:56,645 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:27:56,645 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7F8F131755894368AEB28D20AAC001E4
2016-05-17 22:27:56,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7F8F131755894368AEB28D20AAC001E4
2016-05-17 22:27:56,653 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B9DACBDE3BC249D6A6893046D4CF528E, ordinal: 0
2016-05-17 22:27:56,653 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9DACBDE3BC249D6A6893046D4CF528E, mem on start (mb), free: 69, total: 491, max: 672
2016-05-17 22:27:56,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:56,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:56,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:56,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:56,654 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:56,654 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:27:56,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:56,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:27:56,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:56,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:27:56,656 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7F8F131755894368AEB28D20AAC001E4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:27:56,656 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FE85C7A2AB0C4CDA905D62FD2B11020A, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:27:56,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:56,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7F8F131755894368AEB28D20AAC001E4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:56,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7F8F131755894368AEB28D20AAC001E4
2016-05-17 22:27:56,657 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:56,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7F8F131755894368AEB28D20AAC001E4
2016-05-17 22:27:56,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:56,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:56,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:56,658 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,661 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:27:56,664 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:56,665 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 7F8F131755894368AEB28D20AAC001E4
2016-05-17 22:27:56,665 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: 63FF1BA3E15C4B71B6A95B3D4E76B532
2016-05-17 22:27:56,666 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] 63FF1BA3E15C4B71B6A95B3D4E76B532
2016-05-17 22:27:56,667 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7c70cc18
2016-05-17 22:27:56,668 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7c70cc18
2016-05-17 22:27:56,668 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:27:56,669 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:27:56,669 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@51677b7b; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@60dbf1fd; comparator=cascading.tuple.hadoop.util.TupleComparator@7ec4ab9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:27:56,670 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:27:56,670 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:27:56,670 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:56,704 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:27:56,705 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:27:56,705 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:27:56,705 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:27:56,705 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:27:56,707 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:56,708 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:56,709 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B9DACBDE3BC249D6A6893046D4CF528E, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:56,713 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,714 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9DACBDE3BC249D6A6893046D4CF528E, mem on close (mb), free: 159, total: 485, max: 672
2016-05-17 22:27:56,714 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:27:56,715 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:27:56,715 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:27:56,715 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:27:56,719 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:27:56 Completed running task attempt: attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,726 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524076365_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:56,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524076365_0001_1_00, eventCount=1
2016-05-17 22:27:56,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:56,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B9DACBDE3BC249D6A6893046D4CF528E, taskAttemptId=attempt_1463524076365_0001_1_00_000000_0, startTime=1463524076631, finishTime=1463524076727, timeTaken=96, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524076709, Process_Duration=4, Process_End_Time=1463524076713, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:56,728 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524076365_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:56,728 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:56,728 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524076365_0001_1_00_000000_0
2016-05-17 22:27:56,729 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:56,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:TASK_FINISHED]: vertexName=B9DACBDE3BC249D6A6893046D4CF528E, taskId=task_1463524076365_0001_1_00_000000, startTime=1463524076631, finishTime=1463524076729, timeTaken=98, status=SUCCEEDED, successfulAttemptID=attempt_1463524076365_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524076709, Process_Duration=4, Process_End_Time=1463524076713, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:56,730 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524076365_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:56,730 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] : 1
2016-05-17 22:27:56,730 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,731 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:56,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524076709, Process_Duration=4, Process_End_Time=1463524076713, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:27:56,732 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:56,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:VERTEX_FINISHED]: vertexName=B9DACBDE3BC249D6A6893046D4CF528E, vertexId=vertex_1463524076365_0001_1_00, initRequestedTime=1463524076589, initedTime=1463524076608, startRequestedTime=1463524076611, startedTime=1463524076611, finishTime=1463524076730, timeTaken=119, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524076709, Process_Duration=4, Process_End_Time=1463524076713, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524076631, firstTasksToStart=[ task_1463524076365_0001_1_00_000000 ], lastTaskFinishTime=1463524076727, lastTasksToFinish=[ task_1463524076365_0001_1_00_000000 ], minTaskDuration=96, maxTaskDuration=96, avgTaskDuration=96.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524076365_0001_1_00_000000 ], longestDurationTasks=[ task_1463524076365_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:56,732 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:56,733 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:56,732 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:56,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524076365_0001_00_000001, stoppedTime=1463524076733, exitStatus=0
2016-05-17 22:27:56,733 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:56,734 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:56,734 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A] attempt: attempt_1463524076365_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:27:56,734 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FE85C7A2AB0C4CDA905D62FD2B11020A
2016-05-17 22:27:56,734 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,735 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:56,735 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524076365_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:27:56,735 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524076365_0001_00_000001
2016-05-17 22:27:56,736 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:56,736 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524076365_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:27:56,736 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524076365_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:56,736 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524076365_0001_00_000001, Exiting
2016-05-17 22:27:56,737 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524076365_0001_00_000001
2016-05-17 22:27:56,737 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524076365_0001_00_000001
2016-05-17 22:27:56,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:TASK_STARTED]: vertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, taskId=task_1463524076365_0001_1_01_000000, scheduledTime=1463524076735, launchTime=1463524076735
2016-05-17 22:27:56,737 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524076365_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:56,738 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:27:56,738 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,738 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E] test/(1/1) ...sRemoval/value/NONE/VALID, VertexName: FE85C7A2AB0C4CDA905D62FD2B11020A, VertexParallelism: 1, TaskAttemptID:attempt_1463524076365_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9DACBDE3BC249D6A6893046D4CF528E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8407F60730E044A2A73BE13D5F43337D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:56,738 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524076365_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:56,738 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:56,741 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524076365_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:56,741 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524076365_0001_00_000001 completed successfully
2016-05-17 22:27:56,742 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524076365_0001_00_000002
2016-05-17 22:27:56,742 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:56,742 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:27:56,744 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524076365_0001_00_000002
2016-05-17 22:27:56,745 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:27:56,746 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:56,747 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524076365_0001_00_000002, launchTime=1463524076746
2016-05-17 22:27:56,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:56,747 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524076365_0001_00_000002
2016-05-17 22:27:56,748 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524076365_0001_1_01_000000_0] to container: [container_1463524076365_0001_00_000002]
2016-05-17 22:27:56,749 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:56,749 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524076365_0001_00_000002 given task: attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,750 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524076365_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,750 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524076365_0001_1_01_000000_0] started. Is using containerId: [container_1463524076365_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:27:56,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:56 Starting to run new task attempt: attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, taskAttemptId=attempt_1463524076365_0001_1_01_000000_0, startTime=1463524076749, containerId=container_1463524076365_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524076365_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:27:56,751 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524076365_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:56,751 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,752 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:56,751 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524076365_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:56,752 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:56,752 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:56,752 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:56,753 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:56,753 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:56,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E] test/(1/1) ...sRemoval/value/NONE/VALID, VertexName: FE85C7A2AB0C4CDA905D62FD2B11020A, VertexParallelism: 1, TaskAttemptID:attempt_1463524076365_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9DACBDE3BC249D6A6893046D4CF528E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8407F60730E044A2A73BE13D5F43337D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:56,753 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:56,754 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:56,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:56,755 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524076365_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:27:56,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B9DACBDE3BC249D6A6893046D4CF528E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:27:56,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8407F60730E044A2A73BE13D5F43337D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:27:56,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:56,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:56,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8407F60730E044A2A73BE13D5F43337D
2016-05-17 22:27:56,757 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:27:56,757 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8407F60730E044A2A73BE13D5F43337D
2016-05-17 22:27:56,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:56,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,760 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:27:56,760 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:27:56,760 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,763 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:56,771 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:27:56,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8407F60730E044A2A73BE13D5F43337D
2016-05-17 22:27:56,773 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE85C7A2AB0C4CDA905D62FD2B11020A, ordinal: 1
2016-05-17 22:27:56,773 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE85C7A2AB0C4CDA905D62FD2B11020A, mem on start (mb), free: 152, total: 485, max: 672
2016-05-17 22:27:56,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:56,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:56,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:56,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:56,775 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:56,776 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:27:56,776 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:56,776 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:27:56,777 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:56,777 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:27:56,777 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8407F60730E044A2A73BE13D5F43337D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:27:56,777 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B9DACBDE3BC249D6A6893046D4CF528E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:27:56,778 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:56,778 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,779 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:56,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B9DACBDE3BC249D6A6893046D4CF528E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:56,779 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:27:56,779 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B9DACBDE3BC249D6A6893046D4CF528E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:27:56,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:56,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:56,780 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:27:56,781 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:27:56,782 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:27:56,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:56,784 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,786 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524076365_0001_1_00_000000_0_10021, runDuration: 85223, ]
2016-05-17 22:27:56,787 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:27:56,788 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524076365_0001_1_00_000000_0_10021] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524076788, TimeTaken=1, Rate=0.62 MB/s
2016-05-17 22:27:56,788 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,788 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: 63FF1BA3E15C4B71B6A95B3D4E76B532
2016-05-17 22:27:56,789 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:27:56,789 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/VALID"], id: 8407F60730E044A2A73BE13D5F43337D
2016-05-17 22:27:56,789 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B9DACBDE3BC249D6A6893046D4CF528E] #4 in 2ms
2016-05-17 22:27:56,789 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/VALID"]
2016-05-17 22:27:56,790 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] 63FF1BA3E15C4B71B6A95B3D4E76B532, for 1 inputs
2016-05-17 22:27:56,790 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B9DACBDE3BC249D6A6893046D4CF528E] #4, status:false, isInterrupted:false
2016-05-17 22:27:56,791 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:27:56,791 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:27:56,791 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:56,792 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:27:56,792 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:27:56,792 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,793 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:27:56,793 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE85C7A2AB0C4CDA905D62FD2B11020A, all 1 inputs ready in: 00:00:00.003
2016-05-17 22:27:56,794 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@503cd65e; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@4a1dde79
2016-05-17 22:27:56,796 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:27:56,796 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:27:56,797 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,797 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524076365_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:27:56,798 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,798 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE85C7A2AB0C4CDA905D62FD2B11020A, mem on close (mb), free: 149, total: 485, max: 672
2016-05-17 22:27:56,798 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B9DACBDE3BC249D6A6893046D4CF528E
2016-05-17 22:27:56,799 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:27:56,799 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:27:56,799 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:56,799 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:27:56,800 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:27:56 Completed running task attempt: attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,801 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524076365_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:56,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:56,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, taskAttemptId=attempt_1463524076365_0001_1_01_000000_0, startTime=1463524076749, finishTime=1463524076802, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524076793, Process_Duration=5, Process_End_Time=1463524076798, Read_Duration=0, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:56,803 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524076365_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:56,803 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:56,804 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524076365_0001_1_01_000000_0
2016-05-17 22:27:56,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:56,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:TASK_FINISHED]: vertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, taskId=task_1463524076365_0001_1_01_000000, startTime=1463524076749, finishTime=1463524076804, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1463524076365_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524076793, Process_Duration=5, Process_End_Time=1463524076798, Read_Duration=0, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:56,805 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524076365_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:56,805 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A] : 1
2016-05-17 22:27:56,805 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:56,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:VERTEX_FINISHED]: vertexName=FE85C7A2AB0C4CDA905D62FD2B11020A, vertexId=vertex_1463524076365_0001_1_01, initRequestedTime=1463524076591, initedTime=1463524076603, startRequestedTime=1463524076615, startedTime=1463524076615, finishTime=1463524076805, timeTaken=190, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524076793, Process_Duration=5, Process_End_Time=1463524076798, Read_Duration=0, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1, vertexStats=firstTaskStartTime=1463524076749, firstTasksToStart=[ task_1463524076365_0001_1_01_000000 ], lastTaskFinishTime=1463524076802, lastTasksToFinish=[ task_1463524076365_0001_1_01_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524076365_0001_1_01_000000 ], longestDurationTasks=[ task_1463524076365_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:56,807 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:56,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524076793, Process_Duration=5, Process_End_Time=1463524076798, Read_Duration=0, Tuples_Read=7, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=1]]
2016-05-17 22:27:56,808 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:56,808 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:56,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:56,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524076365_0001_00_000002, stoppedTime=1463524076808, exitStatus=0
2016-05-17 22:27:56,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:56,809 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:56,809 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524076365_0001_1
2016-05-17 22:27:56,808 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524076365_0001_00_000002
2016-05-17 22:27:56,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:56,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524076365_0001_1
2016-05-17 22:27:56,809 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463524076365_0001_1_00 [B9DACBDE3BC249D6A6893046D4CF528E]
2016-05-17 22:27:56,810 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8407F60730E044A2A73BE13D5F43337D for vertex: vertex_1463524076365_0001_1_01 [FE85C7A2AB0C4CDA905D62FD2B11020A]
2016-05-17 22:27:56,810 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524076365_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 22:27:56,810 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524076365_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:56,810 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524076365_0001_00_000002, Exiting
2016-05-17 22:27:56,811 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524076365_0001_00_000002
2016-05-17 22:27:56,811 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524076365_0001_00_000002
2016-05-17 22:27:56,811 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524076365_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:56,812 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524076365_0001_00_000002 completed successfully
2016-05-17 22:27:56,822 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524076365_0001_1, queueSize=0
2016-05-17 22:27:56,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:27:56,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524076365_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524076365_0001_1, startTime=1463524076588, finishTime=1463524076820, timeTaken=232, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2891, FILE_BYTES_WRITTEN=704, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=14, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=21, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1018167296, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=8, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927048153502, Process_Duration=9, Process_End_Time=2927048153511, Read_Duration=2, Tuples_Read=17, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=1
2016-05-17 22:27:56,824 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524076365_0001_1 finished with state: SUCCEEDED
2016-05-17 22:27:56,824 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524076365_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:27:56,824 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524076365_0001_00_000002
2016-05-17 22:27:56,824 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524076365_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:27:56 Completed Dag: dag_1463524076365_0001_1
2016-05-17 22:27:56,825 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524076365_0001_1, dagState=SUCCEEDED
2016-05-17 22:27:56,825 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524076365_0001_1_post
2016-05-17 22:27:56,825 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:56,826 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:27:56,833 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sRemoval/value/NONE/VALID, applicationId=application_1463524076365_0001
2016-05-17 22:27:56,836 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:27:56,836 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:27:56,836 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:27:56,836 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:27:56,836 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:27:56,837 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:27:56,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:27:56,837 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:27:56,838 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:27:56,838 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:27:56,838 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:27:56,839 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:27:56,839 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:27:56,844 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AF540CFED5C9454A8F301E9097354D21/BE4CA40AE11D4717B3E64B585AC31F2E/.tez/application_1463524076365_0001
2016-05-17 22:27:56,845 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/VALID/_temporary
2016-05-17 22:27:56,845 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.507
2016-05-17 22:27:56,915 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:56,917 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:56,977 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 22:27:56,998 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.050
2016-05-17 22:27:56,999 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:56,999 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:56,999 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:57,017 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:57,028 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/VALID
2016-05-17 22:27:57,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:57,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:57,086 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/VALID"]
2016-05-17 22:27:57,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:57,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:57,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:57,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...Removal/value/VALID/VALID
2016-05-17 22:27:57,087 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:57,101 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:57,106 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:57,106 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:57,106 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:57,116 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0/.tez/application_1463524077106_0001 doesn't exist and is created
2016-05-17 22:27:57,133 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:57,133 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:57,134 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0/.tez/application_1463524077106_0001
2016-05-17 22:27:57,138 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524077106_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:57,151 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:57,152 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:57,154 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:57,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:57,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0/.tez/application_1463524077106_0001/localmode-log-dir
2016-05-17 22:27:57,155 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0/.tez/application_1463524077106_0001/localmode-log-dir/history.txt.appattempt_1463524077106_0001_000000, maxErrors=10
2016-05-17 22:27:57,155 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:57,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524077106_0001_000000, appSubmitTime=1463524077138, launchTime=1463524077138
2016-05-17 22:27:57,164 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:57,166 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:57,168 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:57,168 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:57,171 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:55736
2016-05-17 22:27:57,172 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:57,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524077106_0001_000000, startTime=1463524077175
2016-05-17 22:27:57,176 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:57,176 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:57,234 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:57,234 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:57,234 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...Removal/value/VALID/VALID, applicationId=application_1463524077106_0001, dagName=[1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0] test/(1/1) ...Removal/value/VALID/VALID
2016-05-17 22:27:57,257 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0] test/(1/1) ...Removal/value/VALID/VALID
2016-05-17 22:27:57,260 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524077106_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0/.tez/application_1463524077106_0001/localmode-log-dir/dag_1463524077106_0001_1.dot
2016-05-17 22:27:57,262 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0/.tez/application_1463524077106_0001/localmode-log-dir/dag_1463524077106_0001_1-tez-dag.pb.txt
2016-05-17 22:27:57,286 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524077106_0001_1
2016-05-17 22:27:57,286 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:57,287 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0] test/(1/1) ...Removal/value/VALID/VALID
2016-05-17 22:27:57 Running Dag: dag_1463524077106_0001_1
2016-05-17 22:27:57,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524077106_0001_1, submitTime=1463524077258
2016-05-17 22:27:57,308 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:27:57,309 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BE496EA1E0EC4EB598A7BE8A10F564BF
2016-05-17 22:27:57,309 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:27:57,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524077106_0001_1, initTime=1463524077308
2016-05-17 22:27:57,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:27:57,311 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524077106_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:27:57,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:DAG_STARTED]: dagID=dag_1463524077106_0001_1, startTime=1463524077311
2016-05-17 22:27:57,312 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:27:57,313 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:27:57,313 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524077106_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:27:57,313 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BC04E5F505E144CFAB70528D429F4985 : {811ABCCF530A4318940CE9B9331E110F={InputName=811ABCCF530A4318940CE9B9331E110F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:27:57,313 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 811ABCCF530A4318940CE9B9331E110F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:27:57,313 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,313 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,313 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,314 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]. Starting root input initializers: 1
2016-05-17 22:27:57,314 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...Removal/value/VALID/VALID, applicationId=application_1463524077106_0001, dagName=[1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0] test/(1/1) ...Removal/value/VALID/VALID
2016-05-17 22:27:57,315 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:27:57,315 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,318 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:27:57,318 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,318 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,319 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,319 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=23CD4DDBFEEA4A9A8D8830AC1791885E, vertexId=vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:27:57,319 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=23CD4DDBFEEA4A9A8D8830AC1791885E, vertexId=vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,322 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=23CD4DDBFEEA4A9A8D8830AC1791885E, vertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:27:57,322 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=23CD4DDBFEEA4A9A8D8830AC1791885E, vertexId=vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,325 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:57,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, vertexId=vertex_1463524077106_0001_1_01, initRequestedTime=1463524077315, initedTime=1463524077325, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:27:57,326 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:27:57,326 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 811ABCCF530A4318940CE9B9331E110F on vertex vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,328 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:27:57,329 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 811ABCCF530A4318940CE9B9331E110F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:27:57,329 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:27:57,330 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:27:57,332 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:27:57,334 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 811ABCCF530A4318940CE9B9331E110F on vertex vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,334 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:57,334 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {811ABCCF530A4318940CE9B9331E110F=forAllWorkUnits=true, update=[1]}
2016-05-17 22:27:57,334 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] parallelism set to 1
2016-05-17 22:27:57,334 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:57,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524077106_0001_1_00, eventCount=1
2016-05-17 22:27:57,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BC04E5F505E144CFAB70528D429F4985, vertexId=vertex_1463524077106_0001_1_00, initRequestedTime=1463524077313, initedTime=1463524077335, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:27:57,336 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] with distanceFromRoot: 0
2016-05-17 22:27:57,336 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:27:57,336 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,336 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524077106_0001_1_00, startRequestedTime=1463524077336, startedTime=1463524077336
2016-05-17 22:27:57,336 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:57,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:TASK_STARTED]: vertexName=BC04E5F505E144CFAB70528D429F4985, taskId=task_1463524077106_0001_1_00_000000, scheduledTime=1463524077337, launchTime=1463524077337
2016-05-17 22:27:57,337 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077106_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:57,337 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524077106_0001_1_00 for vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] numStartedSources: 1 numSources: 1
2016-05-17 22:27:57,337 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] with distanceFromRoot: 1
2016-05-17 22:27:57,338 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,338 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BC04E5F505E144CFAB70528D429F4985 in vertex: BE496EA1E0EC4EB598A7BE8A10F564BF
2016-05-17 22:27:57,338 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BE496EA1E0EC4EB598A7BE8A10F564BF with 1 source tasks and 1 pending tasks
2016-05-17 22:27:57,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524077106_0001_1_01, startRequestedTime=1463524077338, startedTime=1463524077338
2016-05-17 22:27:57,338 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:57,339 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0] test/(1/1) ...Removal/value/VALID/VALID, VertexName: BC04E5F505E144CFAB70528D429F4985, VertexParallelism: 1, TaskAttemptID:attempt_1463524077106_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=811ABCCF530A4318940CE9B9331E110F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:57,339 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077106_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:57,339 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:57,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:57,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:57,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:57,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:57,341 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:27:57,342 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:27:57,342 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:27:57,343 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524077106_0001_00_000001
2016-05-17 22:27:57,343 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:57,343 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524077106_0001, with dag id: dag_1463524077106_0001_1
2016-05-17 22:27:57,351 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524077106_0001_00_000001
2016-05-17 22:27:57,351 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:57,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:57,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524077106_0001_00_000001, launchTime=1463524077352
2016-05-17 22:27:57,353 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:57,353 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524077106_0001_00_000001
2016-05-17 22:27:57,356 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524077106_0001_1_00_000000_0] to container: [container_1463524077106_0001_00_000001]
2016-05-17 22:27:57,356 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:57,356 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524077106_0001_00_000001 given task: attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,357 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524077106_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524077106_0001_1_00_000000_0] started. Is using containerId: [container_1463524077106_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:27:57 Starting to run new task attempt: attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BC04E5F505E144CFAB70528D429F4985, taskAttemptId=attempt_1463524077106_0001_1_00_000000_0, startTime=1463524077357, containerId=container_1463524077106_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524077106_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:27:57,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:57,359 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077106_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:57,359 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,359 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:57,359 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077106_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:57,359 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:57,360 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:57,360 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:57,360 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:57,360 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:57,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0] test/(1/1) ...Removal/value/VALID/VALID, VertexName: BC04E5F505E144CFAB70528D429F4985, VertexParallelism: 1, TaskAttemptID:attempt_1463524077106_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=811ABCCF530A4318940CE9B9331E110F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:57,361 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:57,361 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:57,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:57,363 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524077106_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:27:57,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:57,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:27:57,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:57,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BE496EA1E0EC4EB598A7BE8A10F564BF
2016-05-17 22:27:57,366 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:27:57,366 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BE496EA1E0EC4EB598A7BE8A10F564BF
2016-05-17 22:27:57,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BE496EA1E0EC4EB598A7BE8A10F564BF
2016-05-17 22:27:57,366 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:57,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=811ABCCF530A4318940CE9B9331E110F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:27:57,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:57,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 811ABCCF530A4318940CE9B9331E110F
2016-05-17 22:27:57,370 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 811ABCCF530A4318940CE9B9331E110F
2016-05-17 22:27:57,375 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:27:57,375 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 811ABCCF530A4318940CE9B9331E110F
2016-05-17 22:27:57,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 811ABCCF530A4318940CE9B9331E110F
2016-05-17 22:27:57,381 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BC04E5F505E144CFAB70528D429F4985, ordinal: 0
2016-05-17 22:27:57,382 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC04E5F505E144CFAB70528D429F4985, mem on start (mb), free: 62, total: 485, max: 672
2016-05-17 22:27:57,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:57,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:57,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:57,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:57,383 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:57,383 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:27:57,383 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:27:57,384 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:57,384 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:57,384 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:27:57,385 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BE496EA1E0EC4EB598A7BE8A10F564BF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:27:57,385 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 811ABCCF530A4318940CE9B9331E110F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:27:57,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:57,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 811ABCCF530A4318940CE9B9331E110F
2016-05-17 22:27:57,386 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:57,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 811ABCCF530A4318940CE9B9331E110F
2016-05-17 22:27:57,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 811ABCCF530A4318940CE9B9331E110F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:57,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:57,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:57,388 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:57,389 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,392 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:57,393 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 811ABCCF530A4318940CE9B9331E110F
2016-05-17 22:27:57,393 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: 604E1CA90DF0400AA84413F8A502869C
2016-05-17 22:27:57,394 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] 604E1CA90DF0400AA84413F8A502869C
2016-05-17 22:27:57,395 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:27:57,396 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@5c99e1fe; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@7ac41a30; comparator=cascading.tuple.hadoop.util.TupleComparator@60938447; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:27:57,396 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:27:57,396 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:27:57,397 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:57,397 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@66e5b079
2016-05-17 22:27:57,414 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@66e5b079
2016-05-17 22:27:57,415 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:27:57,415 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:27:57,425 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:27:57,426 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:27:57,426 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:27:57,426 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:27:57,426 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:27:57,427 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:57,427 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:57,429 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BC04E5F505E144CFAB70528D429F4985, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:57,433 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,433 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC04E5F505E144CFAB70528D429F4985, mem on close (mb), free: 161, total: 490, max: 672
2016-05-17 22:27:57,433 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:27:57,434 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:27:57,434 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:27:57,435 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:27:57,438 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:27:57 Completed running task attempt: attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,445 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524077106_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:57,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524077106_0001_1_00, eventCount=1
2016-05-17 22:27:57,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:57,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BC04E5F505E144CFAB70528D429F4985, taskAttemptId=attempt_1463524077106_0001_1_00_000000_0, startTime=1463524077357, finishTime=1463524077447, timeTaken=90, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524077430, Process_Duration=3, Process_End_Time=1463524077433, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:57,447 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077106_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:57,448 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524077106_0001_1_00_000000_0
2016-05-17 22:27:57,448 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:57,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:57,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:TASK_FINISHED]: vertexName=BC04E5F505E144CFAB70528D429F4985, taskId=task_1463524077106_0001_1_00_000000, startTime=1463524077357, finishTime=1463524077448, timeTaken=91, status=SUCCEEDED, successfulAttemptID=attempt_1463524077106_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524077430, Process_Duration=3, Process_End_Time=1463524077433, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:57,449 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077106_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:57,449 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] : 1
2016-05-17 22:27:57,449 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:57,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:VERTEX_FINISHED]: vertexName=BC04E5F505E144CFAB70528D429F4985, vertexId=vertex_1463524077106_0001_1_00, initRequestedTime=1463524077313, initedTime=1463524077335, startRequestedTime=1463524077336, startedTime=1463524077336, finishTime=1463524077450, timeTaken=114, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524077430, Process_Duration=3, Process_End_Time=1463524077433, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524077357, firstTasksToStart=[ task_1463524077106_0001_1_00_000000 ], lastTaskFinishTime=1463524077447, lastTasksToFinish=[ task_1463524077106_0001_1_00_000000 ], minTaskDuration=90, maxTaskDuration=90, avgTaskDuration=90.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524077106_0001_1_00_000000 ], longestDurationTasks=[ task_1463524077106_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:57,451 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:57,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524077430, Process_Duration=3, Process_End_Time=1463524077433, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:27:57,452 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:57,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524077106_0001_00_000001, stoppedTime=1463524077453, exitStatus=0
2016-05-17 22:27:57,453 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:57,454 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:57,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:57,454 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524077106_0001_00_000001
2016-05-17 22:27:57,454 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:57,455 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] attempt: attempt_1463524077106_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:27:57,456 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BE496EA1E0EC4EB598A7BE8A10F564BF
2016-05-17 22:27:57,456 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,455 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524077106_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:27:57,457 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524077106_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:57,457 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:57,457 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524077106_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:27:57,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:57,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:TASK_STARTED]: vertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, taskId=task_1463524077106_0001_1_01_000000, scheduledTime=1463524077457, launchTime=1463524077457
2016-05-17 22:27:57,458 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077106_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:57,458 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,458 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:27:57,458 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0] test/(1/1) ...Removal/value/VALID/VALID, VertexName: BE496EA1E0EC4EB598A7BE8A10F564BF, VertexParallelism: 1, TaskAttemptID:attempt_1463524077106_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BC04E5F505E144CFAB70528D429F4985, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=23CD4DDBFEEA4A9A8D8830AC1791885E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:57,459 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077106_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:57,459 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524077106_0001_00_000001, Exiting
2016-05-17 22:27:57,459 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524077106_0001_00_000001
2016-05-17 22:27:57,459 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524077106_0001_00_000001
2016-05-17 22:27:57,460 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:57,462 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524077106_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:57,462 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524077106_0001_00_000001 completed successfully
2016-05-17 22:27:57,462 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524077106_0001_00_000002
2016-05-17 22:27:57,463 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:57,465 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524077106_0001_00_000002
2016-05-17 22:27:57,465 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:27:57,466 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:27:57,468 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:57,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:57,469 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524077106_0001_00_000002, launchTime=1463524077468
2016-05-17 22:27:57,470 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524077106_0001_00_000002
2016-05-17 22:27:57,471 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524077106_0001_1_01_000000_0] to container: [container_1463524077106_0001_00_000002]
2016-05-17 22:27:57,471 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:57,472 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524077106_0001_1_01_000000_0] started. Is using containerId: [container_1463524077106_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:27:57,472 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, taskAttemptId=attempt_1463524077106_0001_1_01_000000_0, startTime=1463524077472, containerId=container_1463524077106_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524077106_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:27:57,472 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077106_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:57,473 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077106_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:57,473 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:57,473 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524077106_0001_00_000002 given task: attempt_1463524077106_0001_1_01_000000_0
2016-05-17 22:27:57,473 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524077106_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524077106_0001_1_01_000000_0
2016-05-17 22:27:57 Starting to run new task attempt: attempt_1463524077106_0001_1_01_000000_0
2016-05-17 22:27:57,474 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524077106_0001_1_01_000000_0
2016-05-17 22:27:57,474 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:57,474 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:57,474 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:57,475 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:57,475 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:57,475 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:57,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0] test/(1/1) ...Removal/value/VALID/VALID, VertexName: BE496EA1E0EC4EB598A7BE8A10F564BF, VertexParallelism: 1, TaskAttemptID:attempt_1463524077106_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BC04E5F505E144CFAB70528D429F4985, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=23CD4DDBFEEA4A9A8D8830AC1791885E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:57,476 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:57,476 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524077106_0001_1_01_000000_0
2016-05-17 22:27:57,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:57,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:57,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BC04E5F505E144CFAB70528D429F4985, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:27:57,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:57,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:57,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=23CD4DDBFEEA4A9A8D8830AC1791885E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:27:57,482 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524077106_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:27:57,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:57,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 23CD4DDBFEEA4A9A8D8830AC1791885E
2016-05-17 22:27:57,484 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:27:57,484 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:57,485 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 23CD4DDBFEEA4A9A8D8830AC1791885E
2016-05-17 22:27:57,488 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:27:57,488 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:27:57,488 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,499 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BE496EA1E0EC4EB598A7BE8A10F564BF, ordinal: 1
2016-05-17 22:27:57,499 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE496EA1E0EC4EB598A7BE8A10F564BF, mem on start (mb), free: 155, total: 490, max: 672
2016-05-17 22:27:57,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:57,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:57,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:57,505 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:27:57,505 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 23CD4DDBFEEA4A9A8D8830AC1791885E
2016-05-17 22:27:57,505 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:57,506 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:57,506 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:27:57,506 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:57,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:27:57,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:57,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:27:57,508 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 23CD4DDBFEEA4A9A8D8830AC1791885E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:27:57,508 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BC04E5F505E144CFAB70528D429F4985, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:27:57,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:57,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BC04E5F505E144CFAB70528D429F4985 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:57,512 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:57,513 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:27:57,514 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BC04E5F505E144CFAB70528D429F4985 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:27:57,514 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:27:57,514 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:27:57,514 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:27:57,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:57,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:57,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:57,516 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524077106_0001_1_01_000000_0
2016-05-17 22:27:57,520 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524077106_0001_1_00_000000_0_10026, runDuration: 78110, ]
2016-05-17 22:27:57,521 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:27:57,522 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: 604E1CA90DF0400AA84413F8A502869C
2016-05-17 22:27:57,523 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/VALID"], id: 23CD4DDBFEEA4A9A8D8830AC1791885E
2016-05-17 22:27:57,523 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/VALID"]
2016-05-17 22:27:57,523 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] 604E1CA90DF0400AA84413F8A502869C, for 1 inputs
2016-05-17 22:27:57,524 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524077106_0001_1_00_000000_0_10026] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524077524, TimeTaken=2, Rate=0.31 MB/s
2016-05-17 22:27:57,525 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,525 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:27:57,526 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BC04E5F505E144CFAB70528D429F4985] #5 in 4ms
2016-05-17 22:27:57,526 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BC04E5F505E144CFAB70528D429F4985] #5, status:false, isInterrupted:false
2016-05-17 22:27:57,527 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:27:57,528 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:27:57,528 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:57,528 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:27:57,528 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:27:57,529 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,529 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:27:57,530 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BE496EA1E0EC4EB598A7BE8A10F564BF, all 1 inputs ready in: 00:00:00.005
2016-05-17 22:27:57,532 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@35a1b58a; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@b959288
2016-05-17 22:27:57,534 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:57,541 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:57,548 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:57,551 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:27:57,555 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:27:57,555 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524077106_0001_1_01_000000_0
2016-05-17 22:27:57,556 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524077106_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:27:57,557 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524077106_0001_1_01_000000_0
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:57,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:57,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, taskAttemptId=attempt_1463524077106_0001_1_01_000000_0, startTime=1463524077472, finishTime=1463524077570, timeTaken=98, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524077530, Process_Duration=27, Process_End_Time=1463524077557, Read_Duration=1, Tuples_Read=7
2016-05-17 22:27:57,571 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077106_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:27:57,571 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524077106_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:27:57,571 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:57,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:TASK_FINISHED]: vertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, taskId=task_1463524077106_0001_1_01_000000, startTime=1463524077472, finishTime=1463524077571, timeTaken=99, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-17 22:27:57,572 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077106_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:27:57,572 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] : 1
2016-05-17 22:27:57,572 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] because task failed: task_1463524077106_0001_1_01_000000
2016-05-17 22:27:57,572 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:27:57,573 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:27:57,573 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,573 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=23CD4DDBFEEA4A9A8D8830AC1791885E, vertexId=vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,572 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:57,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:VERTEX_FINISHED]: vertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, vertexId=vertex_1463524077106_0001_1_01, initRequestedTime=1463524077315, initedTime=1463524077325, startRequestedTime=1463524077338, startedTime=1463524077338, finishTime=1463524077574, timeTaken=236, status=FAILED, diagnostics=Task failed, taskId=task_1463524077106_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524077472, firstTasksToStart=[ task_1463524077106_0001_1_01_000000 ], lastTaskFinishTime=1463524077570, lastTasksToFinish=[ task_1463524077106_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:27:57,575 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:57,577 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:27:57,579 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 22:27:57,579 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:27:57,579 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524077106_0001_1
2016-05-17 22:27:57,580 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463524077106_0001_1_00 [BC04E5F505E144CFAB70528D429F4985]
2016-05-17 22:27:57,580 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 23CD4DDBFEEA4A9A8D8830AC1791885E for vertex: vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF]
2016-05-17 22:27:57,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524077530, Process_Duration=27, Process_End_Time=1463524077557, Read_Duration=1, Tuples_Read=7]]
2016-05-17 22:27:57,583 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE496EA1E0EC4EB598A7BE8A10F564BF, mem on close (mb), free: 150, total: 490, max: 672
2016-05-17 22:27:57,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BE496EA1E0EC4EB598A7BE8A10F564BF, index=1
2016-05-17 22:27:57,583 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,584 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524077106_0001_1, queueSize=0
2016-05-17 22:27:57,585 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:27:57,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524077106_0001_1, startTime=1463524077311, finishTime=1463524077581, timeTaken=270, status=FAILED, diagnostics=Vertex failed, vertexName=BE496EA1E0EC4EB598A7BE8A10F564BF, vertexId=vertex_1463524077106_0001_1_01, diagnostics=[Task failed, taskId=task_1463524077106_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runValueAssertions(AppliedAssertionsPlatformTest.java:158)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524077106_0001_1_01 [BE496EA1E0EC4EB598A7BE8A10F564BF] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524077430, Process_Duration=3, Process_End_Time=1463524077433, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:57,585 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524077106_0001_1 finished with state: FAILED
2016-05-17 22:27:57,586 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524077106_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:27:57,584 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:27:57,588 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:27:57,589 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:57,589 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:27:57,589 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:27:57,589 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BE496EA1E0EC4EB598A7BE8A10F564BF, sourceVertex=BC04E5F505E144CFAB70528D429F4985
2016-05-17 22:27:57,589 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BE496EA1E0EC4EB598A7BE8A10F564BF, sourceVertex=23CD4DDBFEEA4A9A8D8830AC1791885E
2016-05-17 22:27:57,589 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524077106_0001_00_000002
2016-05-17 22:27:57,589 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524077106_0001_00_000002
2016-05-17 22:27:57,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077106_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524077106_0001_00_000002, stoppedTime=1463524077594, exitStatus=0
2016-05-17 22:27:57,595 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524077106_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:27:57,595 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524077106_0001_00_000002 completed but with errors
2016-05-17 22:27:57,600 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:57,601 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:57,601 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524077106_0001_1_01_000000_0 failed: true container: container_1463524077106_0001_00_000002 numFailedTAs: 0
2016-05-17 22:27:57 Completed Dag: dag_1463524077106_0001_1
2016-05-17 22:27:57,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524077106_0001_1, dagState=FAILED
2016-05-17 22:27:57,602 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524077106_0001_1_post
2016-05-17 22:27:57,602 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:57,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:27:57,602 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524077106_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:27:57,602 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524077106_0001_00_000002
2016-05-17 22:27:57,602 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077106_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:27:57,603 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077106_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:27:57,605 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...Removal/value/VALID/VALID, applicationId=application_1463524077106_0001
2016-05-17 22:27:57,608 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:27:57,608 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:27:57,608 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:27:57,609 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:27:57,609 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:27:57,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:27:57,610 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:27:57,610 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:27:57,611 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:27:57,611 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:27:57,612 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:27:57,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:27:57,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:27:57,617 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/1E87A05FEC79495BA4133DD1D3ABA3DB/ED8AB0BA92FF4851BA7509CDE5D0F3A0/.tez/application_1463524077106_0001
2016-05-17 22:27:57,618 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping all jobs
2016-05-17 22:27:57,618 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping: (1/1) ...Removal/value/VALID/VALID
2016-05-17 22:27:57,618 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopped all jobs
2016-05-17 22:27:57,619 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/VALID/_temporary
2016-05-17 22:27:57,619 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.533
2016-05-17 22:27:57,620 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/STRICT
2016-05-17 22:27:57,620 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/VALID
2016-05-17 22:27:57,620 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/STRICT/STRICT
2016-05-17 22:27:57,620 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/NONE/VALID
2016-05-17 22:27:57,621 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testValueAssertionsRemoval/value/VALID/STRICT
2016-05-17 22:27:57,666 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:57,667 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:57,735 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-17 22:27:57,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.060
2016-05-17 22:27:57,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:57,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:57,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:57,785 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:57,799 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/STRICT
2016-05-17 22:27:57,885 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:57,886 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:57,886 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/STRICT"]
2016-05-17 22:27:57,886 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:57,887 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:57,887 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:57,888 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...Removal/group/NONE/STRICT
2016-05-17 22:27:57,888 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:57,908 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:57,915 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:57,916 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:57,916 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:57,927 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF/.tez/application_1463524077916_0001 doesn't exist and is created
2016-05-17 22:27:57,947 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:57,947 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:57,947 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF/.tez/application_1463524077916_0001
2016-05-17 22:27:57,951 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524077916_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:57,962 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:57,963 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:57,964 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:57,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:57,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF/.tez/application_1463524077916_0001/localmode-log-dir
2016-05-17 22:27:57,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF/.tez/application_1463524077916_0001/localmode-log-dir/history.txt.appattempt_1463524077916_0001_000000, maxErrors=10
2016-05-17 22:27:57,965 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:57,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524077916_0001_000000, appSubmitTime=1463524077951, launchTime=1463524077951
2016-05-17 22:27:57,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:57,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:57,974 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:57,975 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35693
2016-05-17 22:27:57,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:57,978 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:57,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524077916_0001_000000, startTime=1463524077979
2016-05-17 22:27:57,980 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:57,980 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:58,047 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:58,048 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:58,048 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...Removal/group/NONE/STRICT, applicationId=application_1463524077916_0001, dagName=[168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF] test/(1/1) ...Removal/group/NONE/STRICT
2016-05-17 22:27:58,068 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF] test/(1/1) ...Removal/group/NONE/STRICT
2016-05-17 22:27:58,070 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524077916_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF/.tez/application_1463524077916_0001/localmode-log-dir/dag_1463524077916_0001_1.dot
2016-05-17 22:27:58,070 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF/.tez/application_1463524077916_0001/localmode-log-dir/dag_1463524077916_0001_1-tez-dag.pb.txt
2016-05-17 22:27:58,083 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524077916_0001_1
2016-05-17 22:27:58,083 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,083 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF] test/(1/1) ...Removal/group/NONE/STRICT
2016-05-17 22:27:58 Running Dag: dag_1463524077916_0001_1
2016-05-17 22:27:58,096 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:27:58,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524077916_0001_1, submitTime=1463524078068
2016-05-17 22:27:58,097 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AE6D91ED4FE84B78ACC3F388203C81E4
2016-05-17 22:27:58,098 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:27:58,098 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:27:58,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524077916_0001_1, initTime=1463524078097
2016-05-17 22:27:58,099 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524077916_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:27:58,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:DAG_STARTED]: dagID=dag_1463524077916_0001_1, startTime=1463524078099
2016-05-17 22:27:58,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:27:58,101 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:27:58,102 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524077916_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:27:58,102 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F04D62BA596E4D38AC09DF446C053785 : {80D090965A464A949E627E4B97F8B00D={InputName=80D090965A464A949E627E4B97F8B00D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:27:58,102 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 80D090965A464A949E627E4B97F8B00D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:27:58,102 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,102 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,103 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,103 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...Removal/group/NONE/STRICT, applicationId=application_1463524077916_0001, dagName=[168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF] test/(1/1) ...Removal/group/NONE/STRICT
2016-05-17 22:27:58,103 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]. Starting root input initializers: 1
2016-05-17 22:27:58,104 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:27:58,105 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,106 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 80D090965A464A949E627E4B97F8B00D on vertex vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,107 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:27:58,107 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,108 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,108 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,108 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1E75EED4B5514938BF92E468A3467D4D, vertexId=vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:27:58,108 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:27:58,109 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1E75EED4B5514938BF92E468A3467D4D, vertexId=vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,109 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 80D090965A464A949E627E4B97F8B00D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:27:58,110 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:27:58,110 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:27:58,112 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1E75EED4B5514938BF92E468A3467D4D, vertexName=AE6D91ED4FE84B78ACC3F388203C81E4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:27:58,112 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1E75EED4B5514938BF92E468A3467D4D, vertexId=vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,113 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:27:58,114 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 80D090965A464A949E627E4B97F8B00D on vertex vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AE6D91ED4FE84B78ACC3F388203C81E4, vertexId=vertex_1463524077916_0001_1_01, initRequestedTime=1463524078105, initedTime=1463524078115, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:27:58,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:58,116 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:27:58,116 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:58,116 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {80D090965A464A949E627E4B97F8B00D=forAllWorkUnits=true, update=[1]}
2016-05-17 22:27:58,117 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] parallelism set to 1
2016-05-17 22:27:58,117 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:58,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524077916_0001_1_00, eventCount=1
2016-05-17 22:27:58,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:58,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F04D62BA596E4D38AC09DF446C053785, vertexId=vertex_1463524077916_0001_1_00, initRequestedTime=1463524078102, initedTime=1463524078117, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:27:58,121 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] with distanceFromRoot: 0
2016-05-17 22:27:58,121 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:27:58,122 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,122 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524077916_0001_1_00, startRequestedTime=1463524078122, startedTime=1463524078122
2016-05-17 22:27:58,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:58,123 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:58,123 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:58,123 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:TASK_STARTED]: vertexName=F04D62BA596E4D38AC09DF446C053785, taskId=task_1463524077916_0001_1_00_000000, scheduledTime=1463524078123, launchTime=1463524078123
2016-05-17 22:27:58,124 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077916_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:58,124 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524077916_0001_1_00 for vertex: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4] numStartedSources: 1 numSources: 1
2016-05-17 22:27:58,125 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4] with distanceFromRoot: 1
2016-05-17 22:27:58,125 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,125 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F04D62BA596E4D38AC09DF446C053785 in vertex: AE6D91ED4FE84B78ACC3F388203C81E4
2016-05-17 22:27:58,126 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AE6D91ED4FE84B78ACC3F388203C81E4 with 1 source tasks and 1 pending tasks
2016-05-17 22:27:58,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524077916_0001_1_01, startRequestedTime=1463524078125, startedTime=1463524078125
2016-05-17 22:27:58,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:58,126 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:58,127 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF] test/(1/1) ...Removal/group/NONE/STRICT, VertexName: F04D62BA596E4D38AC09DF446C053785, VertexParallelism: 1, TaskAttemptID:attempt_1463524077916_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=80D090965A464A949E627E4B97F8B00D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AE6D91ED4FE84B78ACC3F388203C81E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:58,127 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077916_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:58,128 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:58,128 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:27:58,129 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:27:58,129 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:27:58,130 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524077916_0001, with dag id: dag_1463524077916_0001_1
2016-05-17 22:27:58,130 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524077916_0001_00_000001
2016-05-17 22:27:58,131 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:58,133 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524077916_0001_00_000001
2016-05-17 22:27:58,133 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:58,134 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:58,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524077916_0001_00_000001, launchTime=1463524078134
2016-05-17 22:27:58,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:58,137 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524077916_0001_00_000001
2016-05-17 22:27:58,140 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524077916_0001_1_00_000000_0] to container: [container_1463524077916_0001_00_000001]
2016-05-17 22:27:58,140 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:58,140 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524077916_0001_00_000001 given task: attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,140 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524077916_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,141 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524077916_0001_1_00_000000_0] started. Is using containerId: [container_1463524077916_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:27:58,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F04D62BA596E4D38AC09DF446C053785, taskAttemptId=attempt_1463524077916_0001_1_00_000000_0, startTime=1463524078140, containerId=container_1463524077916_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524077916_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:27:58,141 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077916_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:58,142 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077916_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:58,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:58 Starting to run new task attempt: attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,143 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,143 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,143 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:58,143 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:58,143 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:58,144 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:58,144 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:58,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF] test/(1/1) ...Removal/group/NONE/STRICT, VertexName: F04D62BA596E4D38AC09DF446C053785, VertexParallelism: 1, TaskAttemptID:attempt_1463524077916_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=80D090965A464A949E627E4B97F8B00D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AE6D91ED4FE84B78ACC3F388203C81E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:58,145 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:58,145 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,145 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:58,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=80D090965A464A949E627E4B97F8B00D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:27:58,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AE6D91ED4FE84B78ACC3F388203C81E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:27:58,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:58,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 80D090965A464A949E627E4B97F8B00D
2016-05-17 22:27:58,150 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 80D090965A464A949E627E4B97F8B00D
2016-05-17 22:27:58,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:58,151 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:58,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AE6D91ED4FE84B78ACC3F388203C81E4
2016-05-17 22:27:58,158 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:27:58,159 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AE6D91ED4FE84B78ACC3F388203C81E4
2016-05-17 22:27:58,159 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524077916_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:27:58,160 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:27:58,161 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 80D090965A464A949E627E4B97F8B00D
2016-05-17 22:27:58,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 80D090965A464A949E627E4B97F8B00D
2016-05-17 22:27:58,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AE6D91ED4FE84B78ACC3F388203C81E4
2016-05-17 22:27:58,167 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F04D62BA596E4D38AC09DF446C053785, ordinal: 0
2016-05-17 22:27:58,168 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F04D62BA596E4D38AC09DF446C053785, mem on start (mb), free: 73, total: 490, max: 672
2016-05-17 22:27:58,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:58,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:58,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:58,168 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:58,168 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:27:58,169 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:58,169 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:27:58,169 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:58,169 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:27:58,169 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 80D090965A464A949E627E4B97F8B00D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:27:58,169 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AE6D91ED4FE84B78ACC3F388203C81E4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:27:58,169 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:58,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 80D090965A464A949E627E4B97F8B00D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:58,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:58,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:58,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 80D090965A464A949E627E4B97F8B00D
2016-05-17 22:27:58,170 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:58,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 80D090965A464A949E627E4B97F8B00D
2016-05-17 22:27:58,170 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:58,171 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,171 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:27:58,171 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@deac744
2016-05-17 22:27:58,173 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@deac744
2016-05-17 22:27:58,173 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:27:58,173 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:27:58,174 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:58,174 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 80D090965A464A949E627E4B97F8B00D
2016-05-17 22:27:58,175 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: C77AB0140F064BAF8312D9A8F9BB11E7
2016-05-17 22:27:58,175 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] C77AB0140F064BAF8312D9A8F9BB11E7
2016-05-17 22:27:58,176 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@4607d187; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@2b4d19fc; comparator=cascading.tuple.hadoop.util.TupleComparator@52e50599; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:27:58,178 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:27:58,178 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:27:58,179 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:58,203 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:27:58,204 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:27:58,204 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:27:58,204 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:27:58,205 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:27:58,206 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:58,206 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:58,207 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F04D62BA596E4D38AC09DF446C053785, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:58,209 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,210 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F04D62BA596E4D38AC09DF446C053785, mem on close (mb), free: 161, total: 490, max: 672
2016-05-17 22:27:58,210 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:27:58,210 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:27:58,210 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:27:58,211 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:27:58,214 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:27:58 Completed running task attempt: attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,219 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524077916_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:58,220 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524078207, Process_Duration=2, Process_End_Time=1463524078209, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:27:58,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524077916_0001_1_00, eventCount=1
2016-05-17 22:27:58,220 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:58,220 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,221 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524077916_0001_00_000001
2016-05-17 22:27:58,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F04D62BA596E4D38AC09DF446C053785, taskAttemptId=attempt_1463524077916_0001_1_00_000000_0, startTime=1463524078140, finishTime=1463524078221, timeTaken=81, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078207, Process_Duration=2, Process_End_Time=1463524078209, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:58,221 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:58,221 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077916_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:58,222 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:58,222 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524077916_0001_1_00_000000_0
2016-05-17 22:27:58,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:58,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:TASK_FINISHED]: vertexName=F04D62BA596E4D38AC09DF446C053785, taskId=task_1463524077916_0001_1_00_000000, startTime=1463524078140, finishTime=1463524078222, timeTaken=82, status=SUCCEEDED, successfulAttemptID=attempt_1463524077916_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078207, Process_Duration=2, Process_End_Time=1463524078209, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:58,223 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077916_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:58,223 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] : 1
2016-05-17 22:27:58,223 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,224 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:58,224 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:VERTEX_FINISHED]: vertexName=F04D62BA596E4D38AC09DF446C053785, vertexId=vertex_1463524077916_0001_1_00, initRequestedTime=1463524078102, initedTime=1463524078117, startRequestedTime=1463524078122, startedTime=1463524078122, finishTime=1463524078223, timeTaken=101, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078207, Process_Duration=2, Process_End_Time=1463524078209, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524078140, firstTasksToStart=[ task_1463524077916_0001_1_00_000000 ], lastTaskFinishTime=1463524078221, lastTasksToFinish=[ task_1463524077916_0001_1_00_000000 ], minTaskDuration=81, maxTaskDuration=81, avgTaskDuration=81.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524077916_0001_1_00_000000 ], longestDurationTasks=[ task_1463524077916_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:58,224 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:58,224 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:58,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524077916_0001_00_000001, stoppedTime=1463524078225, exitStatus=0
2016-05-17 22:27:58,225 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:58,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:58,225 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4] attempt: attempt_1463524077916_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:27:58,225 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AE6D91ED4FE84B78ACC3F388203C81E4
2016-05-17 22:27:58,225 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,226 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:58,226 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524077916_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:27:58,226 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524077916_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:27:58,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:58,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:TASK_STARTED]: vertexName=AE6D91ED4FE84B78ACC3F388203C81E4, taskId=task_1463524077916_0001_1_01_000000, scheduledTime=1463524078226, launchTime=1463524078226
2016-05-17 22:27:58,227 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077916_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:58,227 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:27:58,226 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524077916_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:58,227 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524077916_0001_00_000001, Exiting
2016-05-17 22:27:58,228 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524077916_0001_00_000001
2016-05-17 22:27:58,228 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524077916_0001_00_000001
2016-05-17 22:27:58,227 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,230 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524077916_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:58,231 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524077916_0001_00_000001 completed successfully
2016-05-17 22:27:58,230 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF] test/(1/1) ...Removal/group/NONE/STRICT, VertexName: AE6D91ED4FE84B78ACC3F388203C81E4, VertexParallelism: 1, TaskAttemptID:attempt_1463524077916_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F04D62BA596E4D38AC09DF446C053785, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1E75EED4B5514938BF92E468A3467D4D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:58,231 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077916_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:58,231 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:27:58,232 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:58,233 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524077916_0001_00_000002
2016-05-17 22:27:58,233 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:58,235 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524077916_0001_00_000002
2016-05-17 22:27:58,236 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:27:58,238 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:58,238 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:58,238 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524077916_0001_00_000002, launchTime=1463524078238
2016-05-17 22:27:58,239 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524077916_0001_00_000002
2016-05-17 22:27:58,240 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524077916_0001_1_01_000000_0] to container: [container_1463524077916_0001_00_000002]
2016-05-17 22:27:58,240 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:58,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524077916_0001_1_01_000000_0] started. Is using containerId: [container_1463524077916_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:27:58,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:58,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AE6D91ED4FE84B78ACC3F388203C81E4, taskAttemptId=attempt_1463524077916_0001_1_01_000000_0, startTime=1463524078240, containerId=container_1463524077916_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524077916_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:27:58,241 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077916_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:58,241 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077916_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:58,241 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524077916_0001_00_000002 given task: attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,241 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524077916_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58 Starting to run new task attempt: attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,242 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,242 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,242 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:58,242 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:58,242 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:58,242 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:58,243 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:58,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF] test/(1/1) ...Removal/group/NONE/STRICT, VertexName: AE6D91ED4FE84B78ACC3F388203C81E4, VertexParallelism: 1, TaskAttemptID:attempt_1463524077916_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F04D62BA596E4D38AC09DF446C053785, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1E75EED4B5514938BF92E468A3467D4D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:58,243 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:58,243 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:58,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F04D62BA596E4D38AC09DF446C053785, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:27:58,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1E75EED4B5514938BF92E468A3467D4D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:27:58,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:58,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:58,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1E75EED4B5514938BF92E468A3467D4D
2016-05-17 22:27:58,250 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:27:58,250 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1E75EED4B5514938BF92E468A3467D4D
2016-05-17 22:27:58,252 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:27:58,252 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:27:58,249 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524077916_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:27:58,253 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,252 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:58,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,268 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AE6D91ED4FE84B78ACC3F388203C81E4, ordinal: 1
2016-05-17 22:27:58,268 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE6D91ED4FE84B78ACC3F388203C81E4, mem on start (mb), free: 155, total: 490, max: 672
2016-05-17 22:27:58,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:58,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:58,270 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:27:58,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1E75EED4B5514938BF92E468A3467D4D
2016-05-17 22:27:58,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:58,270 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:58,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:27:58,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:58,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:27:58,272 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:58,272 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:27:58,272 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1E75EED4B5514938BF92E468A3467D4D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:27:58,273 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F04D62BA596E4D38AC09DF446C053785, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:27:58,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:58,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,274 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:58,275 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:27:58,276 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F04D62BA596E4D38AC09DF446C053785 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:27:58,276 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:27:58,276 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:27:58,276 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:27:58,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F04D62BA596E4D38AC09DF446C053785 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:58,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:58,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:58,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:58,278 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,280 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524077916_0001_1_00_000000_0_10033, runDuration: 59661, ]
2016-05-17 22:27:58,281 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:27:58,281 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524077916_0001_1_00_000000_0_10033] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524078281, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 22:27:58,282 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,282 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: C77AB0140F064BAF8312D9A8F9BB11E7
2016-05-17 22:27:58,282 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:27:58,282 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [F04D62BA596E4D38AC09DF446C053785] #6 in 2ms
2016-05-17 22:27:58,282 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/STRICT"], id: 1E75EED4B5514938BF92E468A3467D4D
2016-05-17 22:27:58,283 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/STRICT"]
2016-05-17 22:27:58,283 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] C77AB0140F064BAF8312D9A8F9BB11E7, for 1 inputs
2016-05-17 22:27:58,284 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F04D62BA596E4D38AC09DF446C053785] #6, status:false, isInterrupted:false
2016-05-17 22:27:58,284 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:27:58,285 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:27:58,285 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:58,285 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:27:58,286 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:27:58,286 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,286 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:27:58,286 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AE6D91ED4FE84B78ACC3F388203C81E4, all 1 inputs ready in: 00:00:00.003
2016-05-17 22:27:58,289 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@e58ad87; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5fe8816c
2016-05-17 22:27:58,291 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:27:58,292 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:27:58,292 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,292 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524077916_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:27:58,293 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,294 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE6D91ED4FE84B78ACC3F388203C81E4, mem on close (mb), free: 152, total: 490, max: 672
2016-05-17 22:27:58,294 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F04D62BA596E4D38AC09DF446C053785
2016-05-17 22:27:58,294 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:27:58,295 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:27:58,295 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:58,295 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:27:58,295 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:27:58 Completed running task attempt: attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,296 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524077916_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:58,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:58,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524078287, Process_Duration=6, Process_End_Time=1463524078293, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=1]]
2016-05-17 22:27:58,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AE6D91ED4FE84B78ACC3F388203C81E4, taskAttemptId=attempt_1463524077916_0001_1_01_000000_0, startTime=1463524078240, finishTime=1463524078297, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078287, Process_Duration=6, Process_End_Time=1463524078293, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:58,298 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:58,299 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524077916_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:58,299 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,299 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524077916_0001_1_01_000000_0
2016-05-17 22:27:58,299 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:58,299 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524077916_0001_00_000002
2016-05-17 22:27:58,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:58,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:TASK_FINISHED]: vertexName=AE6D91ED4FE84B78ACC3F388203C81E4, taskId=task_1463524077916_0001_1_01_000000, startTime=1463524078240, finishTime=1463524078299, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1463524077916_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078287, Process_Duration=6, Process_End_Time=1463524078293, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:58,301 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524077916_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:58,301 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4] : 1
2016-05-17 22:27:58,301 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:58,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:VERTEX_FINISHED]: vertexName=AE6D91ED4FE84B78ACC3F388203C81E4, vertexId=vertex_1463524077916_0001_1_01, initRequestedTime=1463524078105, initedTime=1463524078115, startRequestedTime=1463524078125, startedTime=1463524078125, finishTime=1463524078301, timeTaken=176, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078287, Process_Duration=6, Process_End_Time=1463524078293, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1, vertexStats=firstTaskStartTime=1463524078240, firstTasksToStart=[ task_1463524077916_0001_1_01_000000 ], lastTaskFinishTime=1463524078297, lastTasksToFinish=[ task_1463524077916_0001_1_01_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524077916_0001_1_01_000000 ], longestDurationTasks=[ task_1463524077916_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:58,303 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:58,303 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:58,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524077916_0001_00_000002, stoppedTime=1463524078303, exitStatus=0
2016-05-17 22:27:58,303 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:58,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:58,303 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:58,304 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524077916_0001_1
2016-05-17 22:27:58,304 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524077916_0001_1
2016-05-17 22:27:58,304 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463524077916_0001_1_00 [F04D62BA596E4D38AC09DF446C053785]
2016-05-17 22:27:58,304 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1E75EED4B5514938BF92E468A3467D4D for vertex: vertex_1463524077916_0001_1_01 [AE6D91ED4FE84B78ACC3F388203C81E4]
2016-05-17 22:27:58,311 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524077916_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 22:27:58,311 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524077916_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:58,311 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524077916_0001_00_000002, Exiting
2016-05-17 22:27:58,311 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524077916_0001_00_000002
2016-05-17 22:27:58,311 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524077916_0001_00_000002
2016-05-17 22:27:58,314 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524077916_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:58,314 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524077916_0001_00_000002 completed successfully
2016-05-17 22:27:58,317 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524077916_0001_1, queueSize=0
2016-05-17 22:27:58,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:27:58,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524077916_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524077916_0001_1, startTime=1463524078099, finishTime=1463524078315, timeTaken=216, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2891, FILE_BYTES_WRITTEN=704, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=14, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1028653056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=8, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927048156494, Process_Duration=8, Process_End_Time=2927048156502, Read_Duration=2, Tuples_Read=17, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=1
2016-05-17 22:27:58,319 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524077916_0001_1 finished with state: SUCCEEDED
2016-05-17 22:27:58,319 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524077916_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:27:58,319 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524077916_0001_00_000002
2016-05-17 22:27:58,319 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524077916_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:27:58 Completed Dag: dag_1463524077916_0001_1
2016-05-17 22:27:58,320 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524077916_0001_1, dagState=SUCCEEDED
2016-05-17 22:27:58,320 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524077916_0001_1_post
2016-05-17 22:27:58,320 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,320 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:27:58,327 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...Removal/group/NONE/STRICT, applicationId=application_1463524077916_0001
2016-05-17 22:27:58,330 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:27:58,330 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:27:58,330 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:27:58,330 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:27:58,330 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:27:58,331 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:27:58,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:27:58,332 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:27:58,332 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:27:58,332 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:27:58,332 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:27:58,333 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:27:58,333 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:27:58,338 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/168FCF9A609F48F58C2E829BE6CC8F2E/A3C94D9FCFA44BF58D8A0DDC8E0FC4EF/.tez/application_1463524077916_0001
2016-05-17 22:27:58,338 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/STRICT/_temporary
2016-05-17 22:27:58,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.453
2016-05-17 22:27:58,407 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:58,408 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:58,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 22:27:58,469 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-05-17 22:27:58,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:58,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:58,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:58,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:58,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/STRICT
2016-05-17 22:27:58,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:58,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:58,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/STRICT"]
2016-05-17 22:27:58,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:58,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:58,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:58,549 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...emoval/group/VALID/STRICT
2016-05-17 22:27:58,549 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:58,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:58,567 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:58,567 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:58,567 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:58,577 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50/.tez/application_1463524078567_0001 doesn't exist and is created
2016-05-17 22:27:58,594 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:58,594 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:58,595 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50/.tez/application_1463524078567_0001
2016-05-17 22:27:58,599 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524078567_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:58,611 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:58,612 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:58,614 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:58,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:58,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50/.tez/application_1463524078567_0001/localmode-log-dir
2016-05-17 22:27:58,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50/.tez/application_1463524078567_0001/localmode-log-dir/history.txt.appattempt_1463524078567_0001_000000, maxErrors=10
2016-05-17 22:27:58,615 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:58,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524078567_0001_000000, appSubmitTime=1463524078599, launchTime=1463524078599
2016-05-17 22:27:58,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:58,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:58,622 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51690
2016-05-17 22:27:58,623 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:58,625 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:58,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:58,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524078567_0001_000000, startTime=1463524078627
2016-05-17 22:27:58,627 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:58,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:58,695 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:58,695 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:58,695 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emoval/group/VALID/STRICT, applicationId=application_1463524078567_0001, dagName=[DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50] test/(1/1) ...emoval/group/VALID/STRICT
2016-05-17 22:27:58,712 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50] test/(1/1) ...emoval/group/VALID/STRICT
2016-05-17 22:27:58,714 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524078567_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50/.tez/application_1463524078567_0001/localmode-log-dir/dag_1463524078567_0001_1.dot
2016-05-17 22:27:58,714 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50/.tez/application_1463524078567_0001/localmode-log-dir/dag_1463524078567_0001_1-tez-dag.pb.txt
2016-05-17 22:27:58,724 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524078567_0001_1
2016-05-17 22:27:58,725 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,725 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50] test/(1/1) ...emoval/group/VALID/STRICT
2016-05-17 22:27:58 Running Dag: dag_1463524078567_0001_1
2016-05-17 22:27:58,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524078567_0001_1, submitTime=1463524078712
2016-05-17 22:27:58,739 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:27:58,739 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C025885C3DF54BB6B35E4C9758A85327
2016-05-17 22:27:58,740 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:27:58,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524078567_0001_1, initTime=1463524078739
2016-05-17 22:27:58,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:27:58,741 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524078567_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:27:58,742 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emoval/group/VALID/STRICT, applicationId=application_1463524078567_0001, dagName=[DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50] test/(1/1) ...emoval/group/VALID/STRICT
2016-05-17 22:27:58,742 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:DAG_STARTED]: dagID=dag_1463524078567_0001_1, startTime=1463524078741
2016-05-17 22:27:58,742 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:27:58,742 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524078567_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:27:58,743 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CAB32FB6E1DC45C7963D0752330C4765 : {C2D53ACA576C4326A2F3C510E802006A={InputName=C2D53ACA576C4326A2F3C510E802006A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:27:58,743 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C2D53ACA576C4326A2F3C510E802006A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:27:58,743 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:27:58,743 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,743 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,743 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,743 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]. Starting root input initializers: 1
2016-05-17 22:27:58,744 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:27:58,744 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,748 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:27:58,748 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,748 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,749 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,749 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=051EED163D824D4EB2BD246BADF55C8A, vertexId=vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:27:58,749 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=051EED163D824D4EB2BD246BADF55C8A, vertexId=vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,744 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C2D53ACA576C4326A2F3C510E802006A on vertex vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,751 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:27:58,752 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=051EED163D824D4EB2BD246BADF55C8A, vertexName=C025885C3DF54BB6B35E4C9758A85327, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:27:58,752 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C2D53ACA576C4326A2F3C510E802006A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:27:58,752 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=051EED163D824D4EB2BD246BADF55C8A, vertexId=vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,752 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:27:58,753 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:27:58,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:58,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C025885C3DF54BB6B35E4C9758A85327, vertexId=vertex_1463524078567_0001_1_01, initRequestedTime=1463524078744, initedTime=1463524078754, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:27:58,755 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:27:58,755 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:27:58,755 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C2D53ACA576C4326A2F3C510E802006A on vertex vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,756 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:58,756 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C2D53ACA576C4326A2F3C510E802006A=forAllWorkUnits=true, update=[1]}
2016-05-17 22:27:58,756 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] parallelism set to 1
2016-05-17 22:27:58,757 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:58,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524078567_0001_1_00, eventCount=1
2016-05-17 22:27:58,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:58,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAB32FB6E1DC45C7963D0752330C4765, vertexId=vertex_1463524078567_0001_1_00, initRequestedTime=1463524078743, initedTime=1463524078762, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:27:58,763 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] with distanceFromRoot: 0
2016-05-17 22:27:58,763 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:27:58,763 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,764 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524078567_0001_1_00, startRequestedTime=1463524078763, startedTime=1463524078763
2016-05-17 22:27:58,764 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:58,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:58,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:TASK_STARTED]: vertexName=CAB32FB6E1DC45C7963D0752330C4765, taskId=task_1463524078567_0001_1_00_000000, scheduledTime=1463524078765, launchTime=1463524078765
2016-05-17 22:27:58,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:58,765 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524078567_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:58,765 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524078567_0001_1_00 for vertex: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327] numStartedSources: 1 numSources: 1
2016-05-17 22:27:58,765 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327] with distanceFromRoot: 1
2016-05-17 22:27:58,765 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,766 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CAB32FB6E1DC45C7963D0752330C4765 in vertex: C025885C3DF54BB6B35E4C9758A85327
2016-05-17 22:27:58,766 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C025885C3DF54BB6B35E4C9758A85327 with 1 source tasks and 1 pending tasks
2016-05-17 22:27:58,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524078567_0001_1_01, startRequestedTime=1463524078766, startedTime=1463524078766
2016-05-17 22:27:58,766 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:58,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:58,767 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50] test/(1/1) ...emoval/group/VALID/STRICT, VertexName: CAB32FB6E1DC45C7963D0752330C4765, VertexParallelism: 1, TaskAttemptID:attempt_1463524078567_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C2D53ACA576C4326A2F3C510E802006A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C025885C3DF54BB6B35E4C9758A85327, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:58,768 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524078567_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:58,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524078567_0001, with dag id: dag_1463524078567_0001_1
2016-05-17 22:27:58,768 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:58,769 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:27:58,769 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:27:58,770 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:27:58,771 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524078567_0001_00_000001
2016-05-17 22:27:58,772 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:58,773 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524078567_0001_00_000001
2016-05-17 22:27:58,773 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:58,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:58,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524078567_0001_00_000001, launchTime=1463524078774
2016-05-17 22:27:58,774 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:58,775 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524078567_0001_00_000001
2016-05-17 22:27:58,776 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524078567_0001_1_00_000000_0] to container: [container_1463524078567_0001_00_000001]
2016-05-17 22:27:58,777 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:58,777 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524078567_0001_00_000001 given task: attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,777 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524078567_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,777 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524078567_0001_1_00_000000_0] started. Is using containerId: [container_1463524078567_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:27:58,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:58 Starting to run new task attempt: attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,778 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,778 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,778 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:58,778 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:58,778 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:58,778 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:58,779 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:58,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50] test/(1/1) ...emoval/group/VALID/STRICT, VertexName: CAB32FB6E1DC45C7963D0752330C4765, VertexParallelism: 1, TaskAttemptID:attempt_1463524078567_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C2D53ACA576C4326A2F3C510E802006A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C025885C3DF54BB6B35E4C9758A85327, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:58,779 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:58,779 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:58,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,778 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAB32FB6E1DC45C7963D0752330C4765, taskAttemptId=attempt_1463524078567_0001_1_00_000000_0, startTime=1463524078777, containerId=container_1463524078567_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524078567_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:27:58,780 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524078567_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:58,781 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524078567_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:27:58,782 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524078567_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:58,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C2D53ACA576C4326A2F3C510E802006A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:27:58,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C025885C3DF54BB6B35E4C9758A85327, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:27:58,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:58,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C025885C3DF54BB6B35E4C9758A85327
2016-05-17 22:27:58,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:58,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C2D53ACA576C4326A2F3C510E802006A
2016-05-17 22:27:58,787 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C2D53ACA576C4326A2F3C510E802006A
2016-05-17 22:27:58,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,788 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:27:58,789 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C025885C3DF54BB6B35E4C9758A85327
2016-05-17 22:27:58,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C025885C3DF54BB6B35E4C9758A85327
2016-05-17 22:27:58,793 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:58,795 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:27:58,795 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C2D53ACA576C4326A2F3C510E802006A
2016-05-17 22:27:58,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C2D53ACA576C4326A2F3C510E802006A
2016-05-17 22:27:58,801 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAB32FB6E1DC45C7963D0752330C4765, ordinal: 0
2016-05-17 22:27:58,801 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB32FB6E1DC45C7963D0752330C4765, mem on start (mb), free: 67, total: 490, max: 672
2016-05-17 22:27:58,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:58,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:58,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:58,802 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:58,802 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:27:58,802 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:58,802 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:27:58,803 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:58,803 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:27:58,803 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C2D53ACA576C4326A2F3C510E802006A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:27:58,804 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C025885C3DF54BB6B35E4C9758A85327, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:27:58,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:58,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C2D53ACA576C4326A2F3C510E802006A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:58,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C2D53ACA576C4326A2F3C510E802006A
2016-05-17 22:27:58,805 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:58,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C2D53ACA576C4326A2F3C510E802006A
2016-05-17 22:27:58,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:58,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:58,806 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:58,806 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,806 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:27:58,807 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6dacb0dd
2016-05-17 22:27:58,808 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6dacb0dd
2016-05-17 22:27:58,808 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:27:58,808 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:27:58,809 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:58,810 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: C2D53ACA576C4326A2F3C510E802006A
2016-05-17 22:27:58,810 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: 9BC338362C4C4F21AB4725C6D21A42EB
2016-05-17 22:27:58,811 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] 9BC338362C4C4F21AB4725C6D21A42EB
2016-05-17 22:27:58,812 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@52f24202; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1ecbbbd9; comparator=cascading.tuple.hadoop.util.TupleComparator@5199b35a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:27:58,813 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:27:58,813 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:27:58,813 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:58,838 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:27:58,838 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:27:58,839 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:27:58,839 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:27:58,839 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:27:58,840 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:58,840 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:58,841 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAB32FB6E1DC45C7963D0752330C4765, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:58,845 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,845 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB32FB6E1DC45C7963D0752330C4765, mem on close (mb), free: 160, total: 490, max: 672
2016-05-17 22:27:58,845 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:27:58,846 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:27:58,846 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:27:58,846 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:27:58,850 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:27:58 Completed running task attempt: attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,854 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524078567_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:58,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524078567_0001_1_00, eventCount=1
2016-05-17 22:27:58,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:58,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAB32FB6E1DC45C7963D0752330C4765, taskAttemptId=attempt_1463524078567_0001_1_00_000000_0, startTime=1463524078777, finishTime=1463524078855, timeTaken=78, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078841, Process_Duration=4, Process_End_Time=1463524078845, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:58,855 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524078567_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:58,855 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:58,855 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524078567_0001_1_00_000000_0
2016-05-17 22:27:58,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:58,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:TASK_FINISHED]: vertexName=CAB32FB6E1DC45C7963D0752330C4765, taskId=task_1463524078567_0001_1_00_000000, startTime=1463524078777, finishTime=1463524078856, timeTaken=79, status=SUCCEEDED, successfulAttemptID=attempt_1463524078567_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078841, Process_Duration=4, Process_End_Time=1463524078845, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:58,856 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524078567_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:58,857 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] : 1
2016-05-17 22:27:58,857 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524078841, Process_Duration=4, Process_End_Time=1463524078845, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:27:58,857 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:58,857 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,858 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524078567_0001_00_000001
2016-05-17 22:27:58,863 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:58,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:VERTEX_FINISHED]: vertexName=CAB32FB6E1DC45C7963D0752330C4765, vertexId=vertex_1463524078567_0001_1_00, initRequestedTime=1463524078743, initedTime=1463524078762, startRequestedTime=1463524078763, startedTime=1463524078763, finishTime=1463524078857, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078841, Process_Duration=4, Process_End_Time=1463524078845, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524078777, firstTasksToStart=[ task_1463524078567_0001_1_00_000000 ], lastTaskFinishTime=1463524078855, lastTasksToFinish=[ task_1463524078567_0001_1_00_000000 ], minTaskDuration=78, maxTaskDuration=78, avgTaskDuration=78.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524078567_0001_1_00_000000 ], longestDurationTasks=[ task_1463524078567_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:58,864 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:58,864 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:58,864 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327] attempt: attempt_1463524078567_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:27:58,864 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C025885C3DF54BB6B35E4C9758A85327
2016-05-17 22:27:58,864 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524078567_0001_00_000001, stoppedTime=1463524078865, exitStatus=0
2016-05-17 22:27:58,865 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:58,865 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:58,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:TASK_STARTED]: vertexName=C025885C3DF54BB6B35E4C9758A85327, taskId=task_1463524078567_0001_1_01_000000, scheduledTime=1463524078865, launchTime=1463524078865
2016-05-17 22:27:58,866 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524078567_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:58,866 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,866 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524078567_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:27:58,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50] test/(1/1) ...emoval/group/VALID/STRICT, VertexName: C025885C3DF54BB6B35E4C9758A85327, VertexParallelism: 1, TaskAttemptID:attempt_1463524078567_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAB32FB6E1DC45C7963D0752330C4765, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=051EED163D824D4EB2BD246BADF55C8A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:58,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524078567_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:58,866 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:27:58,867 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524078567_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:27:58,867 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:58,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:58,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:58,868 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524078567_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:58,868 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524078567_0001_00_000001, Exiting
2016-05-17 22:27:58,868 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524078567_0001_00_000001
2016-05-17 22:27:58,869 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524078567_0001_00_000001
2016-05-17 22:27:58,871 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524078567_0001_00_000002
2016-05-17 22:27:58,871 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:58,873 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524078567_0001_00_000002
2016-05-17 22:27:58,871 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524078567_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:58,874 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524078567_0001_00_000001 completed successfully
2016-05-17 22:27:58,874 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:27:58,874 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:27:58,876 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:58,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524078567_0001_00_000002, launchTime=1463524078876
2016-05-17 22:27:58,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:58,877 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524078567_0001_00_000002
2016-05-17 22:27:58,878 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524078567_0001_1_01_000000_0] to container: [container_1463524078567_0001_00_000002]
2016-05-17 22:27:58,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:58,879 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524078567_0001_00_000002 given task: attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,879 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524078567_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,879 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524078567_0001_1_01_000000_0] started. Is using containerId: [container_1463524078567_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:27:58,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C025885C3DF54BB6B35E4C9758A85327, taskAttemptId=attempt_1463524078567_0001_1_01_000000_0, startTime=1463524078879, containerId=container_1463524078567_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524078567_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:27:58,880 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524078567_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:58,880 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524078567_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:58,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:58 Starting to run new task attempt: attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,880 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,881 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,881 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:58,881 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:58,882 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:58,882 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:58,882 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:58,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50] test/(1/1) ...emoval/group/VALID/STRICT, VertexName: C025885C3DF54BB6B35E4C9758A85327, VertexParallelism: 1, TaskAttemptID:attempt_1463524078567_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAB32FB6E1DC45C7963D0752330C4765, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=051EED163D824D4EB2BD246BADF55C8A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:58,883 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:58,883 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:58,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CAB32FB6E1DC45C7963D0752330C4765, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:27:58,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:58,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,888 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524078567_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:27:58,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=051EED163D824D4EB2BD246BADF55C8A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:27:58,888 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:58,891 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:27:58,892 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:27:58,892 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:58,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 051EED163D824D4EB2BD246BADF55C8A
2016-05-17 22:27:58,895 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:27:58,895 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 051EED163D824D4EB2BD246BADF55C8A
2016-05-17 22:27:58,901 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C025885C3DF54BB6B35E4C9758A85327, ordinal: 1
2016-05-17 22:27:58,901 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C025885C3DF54BB6B35E4C9758A85327, mem on start (mb), free: 154, total: 490, max: 672
2016-05-17 22:27:58,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:58,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:58,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:58,907 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:27:58,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 051EED163D824D4EB2BD246BADF55C8A
2016-05-17 22:27:58,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:58,907 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:58,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:27:58,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:27:58,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:58,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:58,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:27:58,908 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CAB32FB6E1DC45C7963D0752330C4765, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:27:58,908 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 051EED163D824D4EB2BD246BADF55C8A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:27:58,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:58,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CAB32FB6E1DC45C7963D0752330C4765 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:58,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:58,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:58,910 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:58,910 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:27:58,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CAB32FB6E1DC45C7963D0752330C4765 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:27:58,911 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:27:58,911 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:27:58,911 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:27:58,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:58,913 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,915 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:27:58,915 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524078567_0001_1_00_000000_0_10039, runDuration: 66634, ]
2016-05-17 22:27:58,916 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: 9BC338362C4C4F21AB4725C6D21A42EB
2016-05-17 22:27:58,916 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/STRICT"], id: 051EED163D824D4EB2BD246BADF55C8A
2016-05-17 22:27:58,916 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/STRICT"]
2016-05-17 22:27:58,916 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] 9BC338362C4C4F21AB4725C6D21A42EB, for 1 inputs
2016-05-17 22:27:58,917 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524078567_0001_1_00_000000_0_10039] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524078917, TimeTaken=1, Rate=0.62 MB/s
2016-05-17 22:27:58,917 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,918 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:27:58,918 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CAB32FB6E1DC45C7963D0752330C4765] #7 in 2ms
2016-05-17 22:27:58,918 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CAB32FB6E1DC45C7963D0752330C4765] #7, status:false, isInterrupted:false
2016-05-17 22:27:58,919 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:27:58,919 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:27:58,919 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:58,920 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:27:58,920 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:27:58,920 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,920 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:27:58,921 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C025885C3DF54BB6B35E4C9758A85327, all 1 inputs ready in: 00:00:00.003
2016-05-17 22:27:58,922 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@283046a9; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@796f15db
2016-05-17 22:27:58,924 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:27:58,924 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:27:58,924 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,925 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524078567_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:27:58,925 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,925 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C025885C3DF54BB6B35E4C9758A85327, mem on close (mb), free: 151, total: 490, max: 672
2016-05-17 22:27:58,925 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CAB32FB6E1DC45C7963D0752330C4765
2016-05-17 22:27:58,926 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:27:58,926 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:27:58,926 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:58,927 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:27:58,927 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:27:58 Completed running task attempt: attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,928 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524078567_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:58,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:58,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C025885C3DF54BB6B35E4C9758A85327, taskAttemptId=attempt_1463524078567_0001_1_01_000000_0, startTime=1463524078879, finishTime=1463524078929, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078921, Process_Duration=4, Process_End_Time=1463524078925, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:58,929 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524078567_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:58,929 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:58,929 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524078567_0001_1_01_000000_0
2016-05-17 22:27:58,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:TASK_FINISHED]: vertexName=C025885C3DF54BB6B35E4C9758A85327, taskId=task_1463524078567_0001_1_01_000000, startTime=1463524078879, finishTime=1463524078930, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463524078567_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078921, Process_Duration=4, Process_End_Time=1463524078925, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:27:58,931 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524078567_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:58,932 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327] : 1
2016-05-17 22:27:58,932 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:58,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:58,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:VERTEX_FINISHED]: vertexName=C025885C3DF54BB6B35E4C9758A85327, vertexId=vertex_1463524078567_0001_1_01, initRequestedTime=1463524078744, initedTime=1463524078754, startRequestedTime=1463524078766, startedTime=1463524078766, finishTime=1463524078932, timeTaken=166, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524078921, Process_Duration=4, Process_End_Time=1463524078925, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1, vertexStats=firstTaskStartTime=1463524078879, firstTasksToStart=[ task_1463524078567_0001_1_01_000000 ], lastTaskFinishTime=1463524078929, lastTasksToFinish=[ task_1463524078567_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524078567_0001_1_01_000000 ], longestDurationTasks=[ task_1463524078567_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:58,934 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:58,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524078921, Process_Duration=4, Process_End_Time=1463524078925, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=1]]
2016-05-17 22:27:58,935 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:58,938 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:58,939 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,939 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524078567_0001_00_000002
2016-05-17 22:27:58,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524078567_0001_00_000002, stoppedTime=1463524078939, exitStatus=0
2016-05-17 22:27:58,940 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:58,940 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:58,940 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524078567_0001_1
2016-05-17 22:27:58,940 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524078567_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 22:27:58,940 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:58,940 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524078567_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:58,941 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524078567_0001_00_000002, Exiting
2016-05-17 22:27:58,941 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524078567_0001_00_000002
2016-05-17 22:27:58,941 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524078567_0001_00_000002
2016-05-17 22:27:58,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524078567_0001_1
2016-05-17 22:27:58,943 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 051EED163D824D4EB2BD246BADF55C8A for vertex: vertex_1463524078567_0001_1_01 [C025885C3DF54BB6B35E4C9758A85327]
2016-05-17 22:27:58,943 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524078567_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:58,943 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524078567_0001_00_000002 completed successfully
2016-05-17 22:27:58,952 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463524078567_0001_1_00 [CAB32FB6E1DC45C7963D0752330C4765]
2016-05-17 22:27:58,954 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524078567_0001_1, queueSize=0
2016-05-17 22:27:58,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:27:58,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524078567_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524078567_0001_1, startTime=1463524078741, finishTime=1463524078952, timeTaken=211, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2891, FILE_BYTES_WRITTEN=704, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=14, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1028653056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=8, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927048157762, Process_Duration=8, Process_End_Time=2927048157770, Read_Duration=3, Tuples_Read=17, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=1
2016-05-17 22:27:58,954 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524078567_0001_1 finished with state: SUCCEEDED
2016-05-17 22:27:58,955 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524078567_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:27:58,955 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524078567_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:27:58,955 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524078567_0001_00_000002
2016-05-17 22:27:58 Completed Dag: dag_1463524078567_0001_1
2016-05-17 22:27:58,955 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524078567_0001_1, dagState=SUCCEEDED
2016-05-17 22:27:58,955 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524078567_0001_1_post
2016-05-17 22:27:58,956 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:58,956 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:27:58,963 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emoval/group/VALID/STRICT, applicationId=application_1463524078567_0001
2016-05-17 22:27:58,968 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:27:58,968 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:27:58,968 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:27:58,968 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:27:58,968 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:27:58,970 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:27:58,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:27:58,970 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:27:58,970 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:27:58,971 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:27:58,971 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:27:58,972 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:27:58,972 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:27:58,977 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/DAA4FFB26BB14A519862969A10032044/D0397B048B864769A061DF7B0DC42D50/.tez/application_1463524078567_0001
2016-05-17 22:27:58,977 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/STRICT/_temporary
2016-05-17 22:27:58,978 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.429
2016-05-17 22:27:59,047 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:59,048 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:59,115 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 22:27:59,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-05-17 22:27:59,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:59,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:59,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:59,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:59,165 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/STRICT/STRICT
2016-05-17 22:27:59,208 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:27:59,209 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:27:59,215 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:59,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:59,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/STRICT/STRICT"]
2016-05-17 22:27:59,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:59,217 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:59,217 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:59,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...moval/group/STRICT/STRICT
2016-05-17 22:27:59,219 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:59,231 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:59,235 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:59,236 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:59,236 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:59,245 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9/.tez/application_1463524079236_0001 doesn't exist and is created
2016-05-17 22:27:59,262 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:59,262 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:59,263 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9/.tez/application_1463524079236_0001
2016-05-17 22:27:59,267 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524079236_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:59,278 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:59,279 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:59,281 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:59,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:59,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9/.tez/application_1463524079236_0001/localmode-log-dir
2016-05-17 22:27:59,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9/.tez/application_1463524079236_0001/localmode-log-dir/history.txt.appattempt_1463524079236_0001_000000, maxErrors=10
2016-05-17 22:27:59,283 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:59,284 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524079236_0001_000000, appSubmitTime=1463524079267, launchTime=1463524079267
2016-05-17 22:27:59,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:59,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:59,294 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:59,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:59,295 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45214
2016-05-17 22:27:59,297 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:59,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524079236_0001_000000, startTime=1463524079299
2016-05-17 22:27:59,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:59,299 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:59,363 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:59,363 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:59,363 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...moval/group/STRICT/STRICT, applicationId=application_1463524079236_0001, dagName=[FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9] test/(1/1) ...moval/group/STRICT/STRICT
2016-05-17 22:27:59,381 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9] test/(1/1) ...moval/group/STRICT/STRICT
2016-05-17 22:27:59,382 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524079236_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9/.tez/application_1463524079236_0001/localmode-log-dir/dag_1463524079236_0001_1.dot
2016-05-17 22:27:59,383 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9/.tez/application_1463524079236_0001/localmode-log-dir/dag_1463524079236_0001_1-tez-dag.pb.txt
2016-05-17 22:27:59,399 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524079236_0001_1
2016-05-17 22:27:59,399 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:59,399 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9] test/(1/1) ...moval/group/STRICT/STRICT
2016-05-17 22:27:59 Running Dag: dag_1463524079236_0001_1
2016-05-17 22:27:59,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524079236_0001_1, submitTime=1463524079381
2016-05-17 22:27:59,412 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:27:59,413 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C554266F5FCD46CE84DC61698FD98565
2016-05-17 22:27:59,413 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:27:59,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524079236_0001_1, initTime=1463524079412
2016-05-17 22:27:59,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:27:59,414 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524079236_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:27:59,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:DAG_STARTED]: dagID=dag_1463524079236_0001_1, startTime=1463524079415
2016-05-17 22:27:59,415 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:27:59,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:27:59,415 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524079236_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:27:59,416 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...moval/group/STRICT/STRICT, applicationId=application_1463524079236_0001, dagName=[FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9] test/(1/1) ...moval/group/STRICT/STRICT
2016-05-17 22:27:59,416 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C08980CA7A3843B08215AA6B6E19CBC6 : {28D79E7E8FAD45B0BDD1F4EBF473FE9E={InputName=28D79E7E8FAD45B0BDD1F4EBF473FE9E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:27:59,416 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 28D79E7E8FAD45B0BDD1F4EBF473FE9E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:27:59,416 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,416 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,417 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,417 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]. Starting root input initializers: 1
2016-05-17 22:27:59,418 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:27:59,418 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 28D79E7E8FAD45B0BDD1F4EBF473FE9E on vertex vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,418 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,419 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:27:59,420 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 28D79E7E8FAD45B0BDD1F4EBF473FE9E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:27:59,420 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:27:59,420 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:27:59,422 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:27:59,422 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,422 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,422 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,422 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E68435492676441BBA8933722AE3FA78, vertexId=vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:27:59,422 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E68435492676441BBA8933722AE3FA78, vertexId=vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,423 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:27:59,423 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 28D79E7E8FAD45B0BDD1F4EBF473FE9E on vertex vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,424 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E68435492676441BBA8933722AE3FA78, vertexName=C554266F5FCD46CE84DC61698FD98565, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:27:59,425 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E68435492676441BBA8933722AE3FA78, vertexId=vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C554266F5FCD46CE84DC61698FD98565, vertexId=vertex_1463524079236_0001_1_01, initRequestedTime=1463524079418, initedTime=1463524079427, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:27:59,427 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:27:59,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:59,427 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:59,428 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {28D79E7E8FAD45B0BDD1F4EBF473FE9E=forAllWorkUnits=true, update=[1]}
2016-05-17 22:27:59,428 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] parallelism set to 1
2016-05-17 22:27:59,428 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:27:59,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524079236_0001_1_00, eventCount=1
2016-05-17 22:27:59,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:27:59,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C08980CA7A3843B08215AA6B6E19CBC6, vertexId=vertex_1463524079236_0001_1_00, initRequestedTime=1463524079416, initedTime=1463524079429, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:27:59,429 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] with distanceFromRoot: 0
2016-05-17 22:27:59,430 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:27:59,430 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,430 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:59,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524079236_0001_1_00, startRequestedTime=1463524079430, startedTime=1463524079430
2016-05-17 22:27:59,431 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:59,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:59,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:TASK_STARTED]: vertexName=C08980CA7A3843B08215AA6B6E19CBC6, taskId=task_1463524079236_0001_1_00_000000, scheduledTime=1463524079431, launchTime=1463524079431
2016-05-17 22:27:59,432 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079236_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:59,432 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524079236_0001_1_00 for vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] numStartedSources: 1 numSources: 1
2016-05-17 22:27:59,432 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] with distanceFromRoot: 1
2016-05-17 22:27:59,432 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,432 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524079236_0001, with dag id: dag_1463524079236_0001_1
2016-05-17 22:27:59,435 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C08980CA7A3843B08215AA6B6E19CBC6 in vertex: C554266F5FCD46CE84DC61698FD98565
2016-05-17 22:27:59,435 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C554266F5FCD46CE84DC61698FD98565 with 1 source tasks and 1 pending tasks
2016-05-17 22:27:59,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:27:59,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524079236_0001_1_01, startRequestedTime=1463524079432, startedTime=1463524079432
2016-05-17 22:27:59,436 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:27:59,436 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9] test/(1/1) ...moval/group/STRICT/STRICT, VertexName: C08980CA7A3843B08215AA6B6E19CBC6, VertexParallelism: 1, TaskAttemptID:attempt_1463524079236_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=28D79E7E8FAD45B0BDD1F4EBF473FE9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C554266F5FCD46CE84DC61698FD98565, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:59,436 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079236_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:59,436 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:59,437 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:27:59,439 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:27:59,439 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:27:59,441 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524079236_0001_00_000001
2016-05-17 22:27:59,441 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:59,443 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524079236_0001_00_000001
2016-05-17 22:27:59,443 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:59,445 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:59,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:59,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524079236_0001_00_000001, launchTime=1463524079444
2016-05-17 22:27:59,446 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524079236_0001_00_000001
2016-05-17 22:27:59,447 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524079236_0001_1_00_000000_0] to container: [container_1463524079236_0001_00_000001]
2016-05-17 22:27:59,447 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:59,447 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524079236_0001_00_000001 given task: attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,447 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524079236_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,448 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524079236_0001_1_00_000000_0] started. Is using containerId: [container_1463524079236_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:27:59 Starting to run new task attempt: attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,448 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,448 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:59,448 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:59,448 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:59,448 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:59,448 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:59,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:59,449 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:59,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9] test/(1/1) ...moval/group/STRICT/STRICT, VertexName: C08980CA7A3843B08215AA6B6E19CBC6, VertexParallelism: 1, TaskAttemptID:attempt_1463524079236_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=28D79E7E8FAD45B0BDD1F4EBF473FE9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C554266F5FCD46CE84DC61698FD98565, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:27:59,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C08980CA7A3843B08215AA6B6E19CBC6, taskAttemptId=attempt_1463524079236_0001_1_00_000000_0, startTime=1463524079447, containerId=container_1463524079236_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524079236_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:27:59,449 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:59,449 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079236_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:59,450 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:59,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:59,450 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079236_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:59,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:59,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C554266F5FCD46CE84DC61698FD98565, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:27:59,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:59,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C554266F5FCD46CE84DC61698FD98565
2016-05-17 22:27:59,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=28D79E7E8FAD45B0BDD1F4EBF473FE9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:27:59,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:59,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 28D79E7E8FAD45B0BDD1F4EBF473FE9E
2016-05-17 22:27:59,457 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 28D79E7E8FAD45B0BDD1F4EBF473FE9E
2016-05-17 22:27:59,457 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:59,458 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524079236_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:27:59,459 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:27:59,459 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C554266F5FCD46CE84DC61698FD98565
2016-05-17 22:27:59,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C554266F5FCD46CE84DC61698FD98565
2016-05-17 22:27:59,461 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:27:59,461 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 28D79E7E8FAD45B0BDD1F4EBF473FE9E
2016-05-17 22:27:59,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 28D79E7E8FAD45B0BDD1F4EBF473FE9E
2016-05-17 22:27:59,468 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C08980CA7A3843B08215AA6B6E19CBC6, ordinal: 0
2016-05-17 22:27:59,468 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C08980CA7A3843B08215AA6B6E19CBC6, mem on start (mb), free: 65, total: 490, max: 672
2016-05-17 22:27:59,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:59,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:59,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:59,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:59,469 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:59,470 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:27:59,470 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:59,470 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:27:59,471 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:59,471 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:27:59,471 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 28D79E7E8FAD45B0BDD1F4EBF473FE9E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:27:59,472 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C554266F5FCD46CE84DC61698FD98565, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:27:59,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:59,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 28D79E7E8FAD45B0BDD1F4EBF473FE9E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:59,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:59,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:59,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 28D79E7E8FAD45B0BDD1F4EBF473FE9E
2016-05-17 22:27:59,474 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:59,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 28D79E7E8FAD45B0BDD1F4EBF473FE9E
2016-05-17 22:27:59,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:59,475 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,478 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:27:59,479 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:59,479 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@24ed46f5
2016-05-17 22:27:59,480 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 28D79E7E8FAD45B0BDD1F4EBF473FE9E
2016-05-17 22:27:59,480 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: F96A360E4BF349C7A05EA15D51DCF967
2016-05-17 22:27:59,480 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] F96A360E4BF349C7A05EA15D51DCF967
2016-05-17 22:27:59,481 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@24ed46f5
2016-05-17 22:27:59,481 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:27:59,482 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:27:59,484 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6dd7b3fd; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6c69df61; comparator=cascading.tuple.hadoop.util.TupleComparator@1ac890e2; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:27:59,485 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:27:59,485 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:27:59,485 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:59,511 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:27:59,511 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:27:59,511 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:27:59,511 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:27:59,511 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:27:59,512 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:59,512 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:27:59,512 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C08980CA7A3843B08215AA6B6E19CBC6, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:27:59,515 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,515 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C08980CA7A3843B08215AA6B6E19CBC6, mem on close (mb), free: 158, total: 491, max: 672
2016-05-17 22:27:59,515 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:27:59,516 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:27:59,516 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:27:59,516 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:27:59,519 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:27:59 Completed running task attempt: attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,522 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524079236_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:27:59,523 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1205, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524079512, Process_Duration=3, Process_End_Time=1463524079515, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:27:59,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524079236_0001_1_00, eventCount=1
2016-05-17 22:27:59,523 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:27:59,523 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:59,524 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524079236_0001_00_000001
2016-05-17 22:27:59,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C08980CA7A3843B08215AA6B6E19CBC6, taskAttemptId=attempt_1463524079236_0001_1_00_000000_0, startTime=1463524079447, finishTime=1463524079523, timeTaken=76, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1205, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524079512, Process_Duration=3, Process_End_Time=1463524079515, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:59,524 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:59,524 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079236_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:27:59,524 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524079236_0001_1_00_000000_0
2016-05-17 22:27:59,524 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:59,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:59,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:TASK_FINISHED]: vertexName=C08980CA7A3843B08215AA6B6E19CBC6, taskId=task_1463524079236_0001_1_00_000000, startTime=1463524079447, finishTime=1463524079524, timeTaken=77, status=SUCCEEDED, successfulAttemptID=attempt_1463524079236_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1205, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524079512, Process_Duration=3, Process_End_Time=1463524079515, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:59,526 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079236_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:27:59,526 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] : 1
2016-05-17 22:27:59,526 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:59,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:VERTEX_FINISHED]: vertexName=C08980CA7A3843B08215AA6B6E19CBC6, vertexId=vertex_1463524079236_0001_1_00, initRequestedTime=1463524079416, initedTime=1463524079429, startRequestedTime=1463524079430, startedTime=1463524079430, finishTime=1463524079526, timeTaken=96, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1205, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524079512, Process_Duration=3, Process_End_Time=1463524079515, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524079447, firstTasksToStart=[ task_1463524079236_0001_1_00_000000 ], lastTaskFinishTime=1463524079523, lastTasksToFinish=[ task_1463524079236_0001_1_00_000000 ], minTaskDuration=76, maxTaskDuration=76, avgTaskDuration=76.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524079236_0001_1_00_000000 ], longestDurationTasks=[ task_1463524079236_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:27:59,528 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:27:59,528 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:27:59,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524079236_0001_00_000001, stoppedTime=1463524079528, exitStatus=0
2016-05-17 22:27:59,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:59,528 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:59,529 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] attempt: attempt_1463524079236_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:27:59,529 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C554266F5FCD46CE84DC61698FD98565
2016-05-17 22:27:59,529 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,529 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:27:59,530 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524079236_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:27:59,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:27:59,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:TASK_STARTED]: vertexName=C554266F5FCD46CE84DC61698FD98565, taskId=task_1463524079236_0001_1_01_000000, scheduledTime=1463524079530, launchTime=1463524079530
2016-05-17 22:27:59,530 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079236_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:27:59,530 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,531 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:27:59,531 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9] test/(1/1) ...moval/group/STRICT/STRICT, VertexName: C554266F5FCD46CE84DC61698FD98565, VertexParallelism: 1, TaskAttemptID:attempt_1463524079236_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C08980CA7A3843B08215AA6B6E19CBC6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E68435492676441BBA8933722AE3FA78, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:59,532 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079236_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:27:59,532 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:27:59,533 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524079236_0001_00_000002
2016-05-17 22:27:59,534 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:27:59,535 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524079236_0001_00_000002
2016-05-17 22:27:59,536 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:27:59,536 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524079236_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:27:59,536 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524079236_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:27:59,536 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524079236_0001_00_000001, Exiting
2016-05-17 22:27:59,537 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524079236_0001_00_000001
2016-05-17 22:27:59,537 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524079236_0001_00_000001
2016-05-17 22:27:59,537 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:27:59,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:27:59,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524079236_0001_00_000002, launchTime=1463524079537
2016-05-17 22:27:59,541 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524079236_0001_00_000002
2016-05-17 22:27:59,541 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524079236_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:27:59,541 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524079236_0001_00_000001 completed successfully
2016-05-17 22:27:59,541 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524079236_0001_1_01_000000_0] to container: [container_1463524079236_0001_00_000002]
2016-05-17 22:27:59,542 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:27:59,542 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:27:59,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524079236_0001_1_01_000000_0] started. Is using containerId: [container_1463524079236_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:27:59,543 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C554266F5FCD46CE84DC61698FD98565, taskAttemptId=attempt_1463524079236_0001_1_01_000000_0, startTime=1463524079542, containerId=container_1463524079236_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524079236_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:27:59,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079236_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:27:59,543 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079236_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:27:59,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:27:59,544 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524079236_0001_00_000002 given task: attempt_1463524079236_0001_1_01_000000_0
2016-05-17 22:27:59,544 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524079236_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524079236_0001_1_01_000000_0
2016-05-17 22:27:59 Starting to run new task attempt: attempt_1463524079236_0001_1_01_000000_0
2016-05-17 22:27:59,545 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524079236_0001_1_01_000000_0
2016-05-17 22:27:59,545 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:59,545 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:27:59,545 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:27:59,545 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:27:59,545 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:27:59,545 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:27:59,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9] test/(1/1) ...moval/group/STRICT/STRICT, VertexName: C554266F5FCD46CE84DC61698FD98565, VertexParallelism: 1, TaskAttemptID:attempt_1463524079236_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C08980CA7A3843B08215AA6B6E19CBC6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E68435492676441BBA8933722AE3FA78, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:27:59,546 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:27:59,547 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524079236_0001_1_01_000000_0
2016-05-17 22:27:59,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:27:59,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:59,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C08980CA7A3843B08215AA6B6E19CBC6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:27:59,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:27:59,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,553 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:27:59,553 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:27:59,554 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:59,559 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:27:59,557 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524079236_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:27:59,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E68435492676441BBA8933722AE3FA78, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:27:59,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:27:59,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E68435492676441BBA8933722AE3FA78
2016-05-17 22:27:59,562 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:27:59,562 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E68435492676441BBA8933722AE3FA78
2016-05-17 22:27:59,567 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C554266F5FCD46CE84DC61698FD98565, ordinal: 1
2016-05-17 22:27:59,567 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C554266F5FCD46CE84DC61698FD98565, mem on start (mb), free: 152, total: 491, max: 672
2016-05-17 22:27:59,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:27:59,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:27:59,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:27:59,576 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:27:59,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E68435492676441BBA8933722AE3FA78
2016-05-17 22:27:59,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:27:59,577 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:27:59,577 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:27:59,577 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:27:59,577 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:27:59,577 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:27:59,578 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:27:59,578 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C08980CA7A3843B08215AA6B6E19CBC6, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:27:59,578 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E68435492676441BBA8933722AE3FA78, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:27:59,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:27:59,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C08980CA7A3843B08215AA6B6E19CBC6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:27:59,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:27:59,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:27:59,579 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:27:59,579 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:27:59,579 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C08980CA7A3843B08215AA6B6E19CBC6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:27:59,579 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:27:59,580 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:27:59,580 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:27:59,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:27:59,582 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524079236_0001_1_01_000000_0
2016-05-17 22:27:59,585 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524079236_0001_1_00_000000_0_10044, runDuration: 65116, ]
2016-05-17 22:27:59,586 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:27:59,587 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: F96A360E4BF349C7A05EA15D51DCF967
2016-05-17 22:27:59,588 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/STRICT/STRICT"], id: E68435492676441BBA8933722AE3FA78
2016-05-17 22:27:59,588 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/STRICT/STRICT"]
2016-05-17 22:27:59,589 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524079236_0001_1_00_000000_0_10044] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524079589, TimeTaken=3, Rate=0.21 MB/s
2016-05-17 22:27:59,589 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,589 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] F96A360E4BF349C7A05EA15D51DCF967, for 1 inputs
2016-05-17 22:27:59,590 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:27:59,590 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C08980CA7A3843B08215AA6B6E19CBC6] #8 in 4ms
2016-05-17 22:27:59,590 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C08980CA7A3843B08215AA6B6E19CBC6] #8, status:false, isInterrupted:false
2016-05-17 22:27:59,591 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:27:59,591 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:27:59,591 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:59,592 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:27:59,592 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:27:59,592 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,592 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:27:59,593 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C554266F5FCD46CE84DC61698FD98565, all 1 inputs ready in: 00:00:00.003
2016-05-17 22:27:59,594 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@63cf8142; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@4c2f473f
2016-05-17 22:27:59,596 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:59,602 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:59,609 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:59,616 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:27:59,616 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:27:59,617 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524079236_0001_1_01_000000_0
2016-05-17 22:27:59,617 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524079236_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:27:59,618 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524079236_0001_1_01_000000_0
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:27:59,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:27:59,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C554266F5FCD46CE84DC61698FD98565, taskAttemptId=attempt_1463524079236_0001_1_01_000000_0, startTime=1463524079542, finishTime=1463524079626, timeTaken=84, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524079593, Process_Duration=25, Process_End_Time=1463524079618, Read_Duration=1, Tuples_Read=7
2016-05-17 22:27:59,627 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079236_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:27:59,627 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524079236_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:27:59,627 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:27:59,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:TASK_FINISHED]: vertexName=C554266F5FCD46CE84DC61698FD98565, taskId=task_1463524079236_0001_1_01_000000, startTime=1463524079542, finishTime=1463524079627, timeTaken=85, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-17 22:27:59,628 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079236_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:27:59,628 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] : 1
2016-05-17 22:27:59,628 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:27:59,628 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] because task failed: task_1463524079236_0001_1_01_000000
2016-05-17 22:27:59,628 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:27:59,629 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:27:59,629 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,629 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=E68435492676441BBA8933722AE3FA78, vertexId=vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:27:59,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:VERTEX_FINISHED]: vertexName=C554266F5FCD46CE84DC61698FD98565, vertexId=vertex_1463524079236_0001_1_01, initRequestedTime=1463524079418, initedTime=1463524079427, startRequestedTime=1463524079432, startedTime=1463524079432, finishTime=1463524079630, timeTaken=198, status=FAILED, diagnostics=Task failed, taskId=task_1463524079236_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524079542, firstTasksToStart=[ task_1463524079236_0001_1_01_000000 ], lastTaskFinishTime=1463524079626, lastTasksToFinish=[ task_1463524079236_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:27:59,631 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:27:59,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524079593, Process_Duration=25, Process_End_Time=1463524079618, Read_Duration=1, Tuples_Read=7]]
2016-05-17 22:27:59,631 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C554266F5FCD46CE84DC61698FD98565, mem on close (mb), free: 148, total: 491, max: 672
2016-05-17 22:27:59,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C554266F5FCD46CE84DC61698FD98565, index=1
2016-05-17 22:27:59,632 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,634 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:27:59,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:27:59,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524079236_0001_00_000002, stoppedTime=1463524079634, exitStatus=0
2016-05-17 22:27:59,634 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:27:59,634 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524079236_0001_1_01_000000_0 failed: true container: container_1463524079236_0001_00_000002 numFailedTAs: 0
2016-05-17 22:27:59,635 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 22:27:59,635 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:27:59,635 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:27:59,635 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524079236_0001_1
2016-05-17 22:27:59,635 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463524079236_0001_1_00 [C08980CA7A3843B08215AA6B6E19CBC6]
2016-05-17 22:27:59,635 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: E68435492676441BBA8933722AE3FA78 for vertex: vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565]
2016-05-17 22:27:59,635 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:27:59,636 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:27:59,636 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:27:59,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C554266F5FCD46CE84DC61698FD98565, sourceVertex=C08980CA7A3843B08215AA6B6E19CBC6
2016-05-17 22:27:59,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C554266F5FCD46CE84DC61698FD98565, sourceVertex=E68435492676441BBA8933722AE3FA78
2016-05-17 22:27:59,637 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524079236_0001_00_000002
2016-05-17 22:27:59,637 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524079236_0001_00_000002
2016-05-17 22:27:59,637 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524079236_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:27:59,637 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524079236_0001_00_000002 completed but with errors
2016-05-17 22:27:59,638 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524079236_0001_1, queueSize=0
2016-05-17 22:27:59,639 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:27:59,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079236_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524079236_0001_1, startTime=1463524079415, finishTime=1463524079636, timeTaken=221, status=FAILED, diagnostics=Vertex failed, vertexName=C554266F5FCD46CE84DC61698FD98565, vertexId=vertex_1463524079236_0001_1_01, diagnostics=[Task failed, taskId=task_1463524079236_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524079236_0001_1_01 [C554266F5FCD46CE84DC61698FD98565] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=1205, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524079512, Process_Duration=3, Process_End_Time=1463524079515, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:27:59,639 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524079236_0001_1 finished with state: FAILED
2016-05-17 22:27:59,640 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524079236_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:27:59,640 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524079236_0001_00_000002
2016-05-17 22:27:59,640 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524079236_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:27:59,641 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079236_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:27:59 Completed Dag: dag_1463524079236_0001_1
2016-05-17 22:27:59,641 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524079236_0001_1, dagState=FAILED
2016-05-17 22:27:59,642 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524079236_0001_1_post
2016-05-17 22:27:59,642 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:27:59,642 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:27:59,642 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079236_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:27:59,646 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...moval/group/STRICT/STRICT, applicationId=application_1463524079236_0001
2016-05-17 22:27:59,650 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:27:59,650 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:27:59,650 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:27:59,650 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:27:59,650 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:27:59,650 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:27:59,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:27:59,656 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:27:59,656 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:27:59,656 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:27:59,656 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:27:59,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:27:59,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:27:59,661 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/FDE71EC7A7B5438086496AE26F31DBAF/B3FCDD6CF7FF4A48AB6559EF9951F1F9/.tez/application_1463524079236_0001
2016-05-17 22:27:59,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping all jobs
2016-05-17 22:27:59,661 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping: (1/1) ...moval/group/STRICT/STRICT
2016-05-17 22:27:59,662 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopped all jobs
2016-05-17 22:27:59,662 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/STRICT/STRICT/_temporary
2016-05-17 22:27:59,663 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.447
2016-05-17 22:27:59,704 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:27:59,705 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:27:59,756 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 22:27:59,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-17 22:27:59,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:59,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:27:59,759 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:27:59,775 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:27:59,783 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/VALID
2016-05-17 22:27:59,834 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:27:59,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:27:59,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/VALID"]
2016-05-17 22:27:59,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:27:59,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:27:59,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:27:59,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...sRemoval/group/NONE/VALID
2016-05-17 22:27:59,836 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:27:59,849 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:27:59,854 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:59,854 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:27:59,855 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:27:59,864 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF/.tez/application_1463524079854_0001 doesn't exist and is created
2016-05-17 22:27:59,881 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:27:59,882 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:27:59,882 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF/.tez/application_1463524079854_0001
2016-05-17 22:27:59,886 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524079854_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:27:59,897 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:27:59,898 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:27:59,901 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:27:59,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:27:59,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF/.tez/application_1463524079854_0001/localmode-log-dir
2016-05-17 22:27:59,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF/.tez/application_1463524079854_0001/localmode-log-dir/history.txt.appattempt_1463524079854_0001_000000, maxErrors=10
2016-05-17 22:27:59,902 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:27:59,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524079854_0001_000000, appSubmitTime=1463524079885, launchTime=1463524079885
2016-05-17 22:27:59,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:27:59,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:27:59,910 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42170
2016-05-17 22:27:59,911 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:27:59,912 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:27:59,912 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:27:59,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524079854_0001_000000, startTime=1463524079914
2016-05-17 22:27:59,914 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:27:59,914 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:27:59,982 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:27:59,982 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:27:59,982 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...sRemoval/group/NONE/VALID, applicationId=application_1463524079854_0001, dagName=[4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF] test/(1/1) ...sRemoval/group/NONE/VALID
2016-05-17 22:28:00,000 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF] test/(1/1) ...sRemoval/group/NONE/VALID
2016-05-17 22:28:00,001 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524079854_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF/.tez/application_1463524079854_0001/localmode-log-dir/dag_1463524079854_0001_1.dot
2016-05-17 22:28:00,002 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF/.tez/application_1463524079854_0001/localmode-log-dir/dag_1463524079854_0001_1-tez-dag.pb.txt
2016-05-17 22:28:00,012 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524079854_0001_1
2016-05-17 22:28:00,012 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:00,012 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF] test/(1/1) ...sRemoval/group/NONE/VALID
2016-05-17 22:28:00 Running Dag: dag_1463524079854_0001_1
2016-05-17 22:28:00,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:28:00,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524079854_0001_1, submitTime=1463524080000
2016-05-17 22:28:00,029 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DFE3638F52B4414285A7D1A54F5B8D20
2016-05-17 22:28:00,029 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:28:00,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:28:00,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524079854_0001_1, initTime=1463524080028
2016-05-17 22:28:00,030 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524079854_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:28:00,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:DAG_STARTED]: dagID=dag_1463524079854_0001_1, startTime=1463524080030
2016-05-17 22:28:00,030 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:28:00,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:28:00,030 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524079854_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:28:00,031 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E41DBFCEB8FC4068BDAEA63828ACADDF : {E219092510104BF88274A76DFDA3F38C={InputName=E219092510104BF88274A76DFDA3F38C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:28:00,031 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E219092510104BF88274A76DFDA3F38C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:28:00,031 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,031 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,031 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,032 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]. Starting root input initializers: 1
2016-05-17 22:28:00,032 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:28:00,033 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,034 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E219092510104BF88274A76DFDA3F38C on vertex vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,037 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...sRemoval/group/NONE/VALID, applicationId=application_1463524079854_0001, dagName=[4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF] test/(1/1) ...sRemoval/group/NONE/VALID
2016-05-17 22:28:00,037 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:28:00,037 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,038 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,038 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,039 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A5AE188C0C77405C80DC11476990E811, vertexId=vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:28:00,039 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A5AE188C0C77405C80DC11476990E811, vertexId=vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,038 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:28:00,041 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E219092510104BF88274A76DFDA3F38C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:28:00,043 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:28:00,043 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A5AE188C0C77405C80DC11476990E811, vertexName=DFE3638F52B4414285A7D1A54F5B8D20, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:28:00,045 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A5AE188C0C77405C80DC11476990E811, vertexId=vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,044 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:28:00,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:00,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DFE3638F52B4414285A7D1A54F5B8D20, vertexId=vertex_1463524079854_0001_1_01, initRequestedTime=1463524080033, initedTime=1463524080047, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:28:00,049 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:28:00,050 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:28:00,051 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E219092510104BF88274A76DFDA3F38C on vertex vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,051 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:00,052 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E219092510104BF88274A76DFDA3F38C=forAllWorkUnits=true, update=[1]}
2016-05-17 22:28:00,052 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] parallelism set to 1
2016-05-17 22:28:00,052 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:00,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524079854_0001_1_00, eventCount=1
2016-05-17 22:28:00,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, vertexId=vertex_1463524079854_0001_1_00, initRequestedTime=1463524080031, initedTime=1463524080054, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:28:00,054 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524079854_0001, with dag id: dag_1463524079854_0001_1
2016-05-17 22:28:00,054 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] with distanceFromRoot: 0
2016-05-17 22:28:00,055 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:28:00,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:00,055 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,055 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,055 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:00,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524079854_0001_1_00, startRequestedTime=1463524080055, startedTime=1463524080055
2016-05-17 22:28:00,056 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:00,059 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:00,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:TASK_STARTED]: vertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, taskId=task_1463524079854_0001_1_00_000000, scheduledTime=1463524080059, launchTime=1463524080059
2016-05-17 22:28:00,059 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079854_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:00,060 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524079854_0001_1_00 for vertex: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20] numStartedSources: 1 numSources: 1
2016-05-17 22:28:00,060 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20] with distanceFromRoot: 1
2016-05-17 22:28:00,061 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,062 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E41DBFCEB8FC4068BDAEA63828ACADDF in vertex: DFE3638F52B4414285A7D1A54F5B8D20
2016-05-17 22:28:00,062 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DFE3638F52B4414285A7D1A54F5B8D20 with 1 source tasks and 1 pending tasks
2016-05-17 22:28:00,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524079854_0001_1_01, startRequestedTime=1463524080062, startedTime=1463524080062
2016-05-17 22:28:00,063 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:00,063 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:00,064 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF] test/(1/1) ...sRemoval/group/NONE/VALID, VertexName: E41DBFCEB8FC4068BDAEA63828ACADDF, VertexParallelism: 1, TaskAttemptID:attempt_1463524079854_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E219092510104BF88274A76DFDA3F38C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DFE3638F52B4414285A7D1A54F5B8D20, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:00,064 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079854_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:00,064 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:00,065 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:28:00,066 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:28:00,066 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:28:00,067 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524079854_0001_00_000001
2016-05-17 22:28:00,067 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:00,068 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524079854_0001_00_000001
2016-05-17 22:28:00,068 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:00,069 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:00,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524079854_0001_00_000001, launchTime=1463524080069
2016-05-17 22:28:00,069 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:00,070 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524079854_0001_00_000001
2016-05-17 22:28:00,071 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524079854_0001_1_00_000000_0] to container: [container_1463524079854_0001_00_000001]
2016-05-17 22:28:00,071 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:00,072 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524079854_0001_1_00_000000_0] started. Is using containerId: [container_1463524079854_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:28:00,072 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:00,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, taskAttemptId=attempt_1463524079854_0001_1_00_000000_0, startTime=1463524080072, containerId=container_1463524079854_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524079854_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:28:00,072 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079854_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:00,073 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079854_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:00,073 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524079854_0001_00_000001 given task: attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00,073 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524079854_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00 Starting to run new task attempt: attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00,073 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00,074 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:00,074 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:00,074 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:00,074 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:00,074 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:00,074 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:00,074 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF] test/(1/1) ...sRemoval/group/NONE/VALID, VertexName: E41DBFCEB8FC4068BDAEA63828ACADDF, VertexParallelism: 1, TaskAttemptID:attempt_1463524079854_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E219092510104BF88274A76DFDA3F38C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DFE3638F52B4414285A7D1A54F5B8D20, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:00,075 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:00,075 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:00,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E219092510104BF88274A76DFDA3F38C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:28:00,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DFE3638F52B4414285A7D1A54F5B8D20, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:28:00,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:00,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DFE3638F52B4414285A7D1A54F5B8D20
2016-05-17 22:28:00,080 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:00,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:00,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E219092510104BF88274A76DFDA3F38C
2016-05-17 22:28:00,082 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E219092510104BF88274A76DFDA3F38C
2016-05-17 22:28:00,083 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:28:00,085 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DFE3638F52B4414285A7D1A54F5B8D20
2016-05-17 22:28:00,086 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DFE3638F52B4414285A7D1A54F5B8D20
2016-05-17 22:28:00,086 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524079854_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:28:00,088 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:28:00,089 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E219092510104BF88274A76DFDA3F38C
2016-05-17 22:28:00,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E219092510104BF88274A76DFDA3F38C
2016-05-17 22:28:00,093 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E41DBFCEB8FC4068BDAEA63828ACADDF, ordinal: 0
2016-05-17 22:28:00,094 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E41DBFCEB8FC4068BDAEA63828ACADDF, mem on start (mb), free: 72, total: 491, max: 672
2016-05-17 22:28:00,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:00,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:00,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:00,095 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:00,096 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:28:00,096 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:00,096 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:28:00,097 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:00,097 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:28:00,098 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E219092510104BF88274A76DFDA3F38C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:28:00,098 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DFE3638F52B4414285A7D1A54F5B8D20, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:28:00,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:00,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E219092510104BF88274A76DFDA3F38C
2016-05-17 22:28:00,099 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:00,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E219092510104BF88274A76DFDA3F38C
2016-05-17 22:28:00,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E219092510104BF88274A76DFDA3F38C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:00,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:00,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:00,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:00,101 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00,102 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:28:00,103 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@196008b8
2016-05-17 22:28:00,104 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:00,105 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@196008b8
2016-05-17 22:28:00,105 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:28:00,105 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:28:00,105 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: E219092510104BF88274A76DFDA3F38C
2016-05-17 22:28:00,105 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: 39113B9E574B4335996C875A44D0C9F0
2016-05-17 22:28:00,106 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] 39113B9E574B4335996C875A44D0C9F0
2016-05-17 22:28:00,107 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3b690c36; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6da0cbb2; comparator=cascading.tuple.hadoop.util.TupleComparator@579d4b58; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:28:00,108 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:28:00,108 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:28:00,108 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:00,134 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:28:00,135 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:28:00,135 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:28:00,136 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:28:00,136 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:28:00,137 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:00,137 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:00,137 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E41DBFCEB8FC4068BDAEA63828ACADDF, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:28:00,141 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00,141 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E41DBFCEB8FC4068BDAEA63828ACADDF, mem on close (mb), free: 155, total: 490, max: 672
2016-05-17 22:28:00,141 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:28:00,142 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:28:00,142 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:28:00,142 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:28:00,145 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:28:00 Completed running task attempt: attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00,149 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524079854_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:00,152 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524079854_0001_1_00, eventCount=1
2016-05-17 22:28:00,152 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, taskAttemptId=attempt_1463524079854_0001_1_00_000000_0, startTime=1463524080072, finishTime=1463524080152, timeTaken=80, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080137, Process_Duration=4, Process_End_Time=1463524080141, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:00,153 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079854_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:00,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:00,153 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:00,153 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524079854_0001_1_00_000000_0
2016-05-17 22:28:00,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:00,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:TASK_FINISHED]: vertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, taskId=task_1463524079854_0001_1_00_000000, startTime=1463524080072, finishTime=1463524080153, timeTaken=81, status=SUCCEEDED, successfulAttemptID=attempt_1463524079854_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080137, Process_Duration=4, Process_End_Time=1463524080141, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:00,154 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079854_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:00,154 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] : 1
2016-05-17 22:28:00,154 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:00,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:VERTEX_FINISHED]: vertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, vertexId=vertex_1463524079854_0001_1_00, initRequestedTime=1463524080031, initedTime=1463524080054, startRequestedTime=1463524080055, startedTime=1463524080055, finishTime=1463524080154, timeTaken=99, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080137, Process_Duration=4, Process_End_Time=1463524080141, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524080072, firstTasksToStart=[ task_1463524079854_0001_1_00_000000 ], lastTaskFinishTime=1463524080152, lastTasksToFinish=[ task_1463524079854_0001_1_00_000000 ], minTaskDuration=80, maxTaskDuration=80, avgTaskDuration=80.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524079854_0001_1_00_000000 ], longestDurationTasks=[ task_1463524079854_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:00,159 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:00,159 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:00,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524080137, Process_Duration=4, Process_End_Time=1463524080141, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:28:00,159 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20] attempt: attempt_1463524079854_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:28:00,159 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:00,159 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DFE3638F52B4414285A7D1A54F5B8D20
2016-05-17 22:28:00,160 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,160 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:00,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524079854_0001_00_000001, stoppedTime=1463524080160, exitStatus=0
2016-05-17 22:28:00,160 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524079854_0001_00_000001
2016-05-17 22:28:00,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:00,160 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524079854_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:28:00,160 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:00,161 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:00,161 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524079854_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:28:00,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:00,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:TASK_STARTED]: vertexName=DFE3638F52B4414285A7D1A54F5B8D20, taskId=task_1463524079854_0001_1_01_000000, scheduledTime=1463524080161, launchTime=1463524080161
2016-05-17 22:28:00,161 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524079854_0001_00_000001, Exiting
2016-05-17 22:28:00,161 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524079854_0001_00_000001
2016-05-17 22:28:00,162 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524079854_0001_00_000001
2016-05-17 22:28:00,161 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079854_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:00,162 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,162 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524079854_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:28:00,162 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF] test/(1/1) ...sRemoval/group/NONE/VALID, VertexName: DFE3638F52B4414285A7D1A54F5B8D20, VertexParallelism: 1, TaskAttemptID:attempt_1463524079854_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A5AE188C0C77405C80DC11476990E811, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:00,163 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079854_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:00,163 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:28:00,163 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:00,163 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524079854_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:28:00,164 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524079854_0001_00_000001 completed successfully
2016-05-17 22:28:00,164 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524079854_0001_00_000002
2016-05-17 22:28:00,164 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:00,164 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:28:00,166 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524079854_0001_00_000002
2016-05-17 22:28:00,167 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:28:00,169 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:00,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:00,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524079854_0001_00_000002, launchTime=1463524080169
2016-05-17 22:28:00,172 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524079854_0001_00_000002
2016-05-17 22:28:00,173 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524079854_0001_1_01_000000_0] to container: [container_1463524079854_0001_00_000002]
2016-05-17 22:28:00,173 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:00,173 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524079854_0001_00_000002 given task: attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,174 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524079854_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,174 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524079854_0001_1_01_000000_0] started. Is using containerId: [container_1463524079854_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:28:00,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DFE3638F52B4414285A7D1A54F5B8D20, taskAttemptId=attempt_1463524079854_0001_1_01_000000_0, startTime=1463524080173, containerId=container_1463524079854_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524079854_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:28:00,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:00,174 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079854_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:00,174 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079854_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:00 Starting to run new task attempt: attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,175 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,175 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:00,176 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:00,176 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:00,176 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:00,176 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:00,176 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:00,177 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF] test/(1/1) ...sRemoval/group/NONE/VALID, VertexName: DFE3638F52B4414285A7D1A54F5B8D20, VertexParallelism: 1, TaskAttemptID:attempt_1463524079854_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A5AE188C0C77405C80DC11476990E811, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:00,177 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:00,178 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:00,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,180 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E41DBFCEB8FC4068BDAEA63828ACADDF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:28:00,181 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:00,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,182 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524079854_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:28:00,183 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:28:00,184 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:28:00,184 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A5AE188C0C77405C80DC11476990E811, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:28:00,189 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:00,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:00,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A5AE188C0C77405C80DC11476990E811
2016-05-17 22:28:00,192 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:28:00,193 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A5AE188C0C77405C80DC11476990E811
2016-05-17 22:28:00,202 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DFE3638F52B4414285A7D1A54F5B8D20, ordinal: 1
2016-05-17 22:28:00,202 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DFE3638F52B4414285A7D1A54F5B8D20, mem on start (mb), free: 149, total: 490, max: 672
2016-05-17 22:28:00,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:00,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:00,210 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:28:00,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A5AE188C0C77405C80DC11476990E811
2016-05-17 22:28:00,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:00,211 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:00,211 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:28:00,211 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:28:00,211 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:00,211 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:00,212 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:28:00,212 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E41DBFCEB8FC4068BDAEA63828ACADDF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:28:00,212 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A5AE188C0C77405C80DC11476990E811, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:28:00,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:00,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E41DBFCEB8FC4068BDAEA63828ACADDF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:00,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:00,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:00,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,213 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:00,214 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:28:00,214 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E41DBFCEB8FC4068BDAEA63828ACADDF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:28:00,214 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:28:00,214 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:28:00,214 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:28:00,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:00,217 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,219 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524079854_0001_1_00_000000_0_10050, runDuration: 68718, ]
2016-05-17 22:28:00,220 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:28:00,221 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: 39113B9E574B4335996C875A44D0C9F0
2016-05-17 22:28:00,221 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/VALID"], id: A5AE188C0C77405C80DC11476990E811
2016-05-17 22:28:00,222 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/VALID"]
2016-05-17 22:28:00,222 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] 39113B9E574B4335996C875A44D0C9F0, for 1 inputs
2016-05-17 22:28:00,222 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524079854_0001_1_00_000000_0_10050] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524080222, TimeTaken=2, Rate=0.31 MB/s
2016-05-17 22:28:00,223 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,223 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:28:00,224 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E41DBFCEB8FC4068BDAEA63828ACADDF] #9 in 4ms
2016-05-17 22:28:00,224 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E41DBFCEB8FC4068BDAEA63828ACADDF] #9, status:false, isInterrupted:false
2016-05-17 22:28:00,224 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:28:00,225 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:28:00,225 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:00,225 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:28:00,226 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:28:00,226 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,226 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:28:00,226 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DFE3638F52B4414285A7D1A54F5B8D20, all 1 inputs ready in: 00:00:00.004
2016-05-17 22:28:00,229 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@f89699f; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6754db1
2016-05-17 22:28:00,231 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:28:00,232 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:28:00,233 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,233 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524079854_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:28:00,234 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,235 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DFE3638F52B4414285A7D1A54F5B8D20, mem on close (mb), free: 146, total: 490, max: 672
2016-05-17 22:28:00,235 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E41DBFCEB8FC4068BDAEA63828ACADDF
2016-05-17 22:28:00,235 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:28:00,236 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:28:00,236 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:00,236 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:28:00,236 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:28:00 Completed running task attempt: attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,239 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524079854_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:00,240 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524080226, Process_Duration=8, Process_End_Time=1463524080234, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=1]]
2016-05-17 22:28:00,240 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:00,240 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:00,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DFE3638F52B4414285A7D1A54F5B8D20, taskAttemptId=attempt_1463524079854_0001_1_01_000000_0, startTime=1463524080173, finishTime=1463524080240, timeTaken=67, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080226, Process_Duration=8, Process_End_Time=1463524080234, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:28:00,241 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524079854_0001_00_000002
2016-05-17 22:28:00,241 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524079854_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:00,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:00,241 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524079854_0001_1_01_000000_0
2016-05-17 22:28:00,242 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:00,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:TASK_FINISHED]: vertexName=DFE3638F52B4414285A7D1A54F5B8D20, taskId=task_1463524079854_0001_1_01_000000, startTime=1463524080173, finishTime=1463524080241, timeTaken=68, status=SUCCEEDED, successfulAttemptID=attempt_1463524079854_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080226, Process_Duration=8, Process_End_Time=1463524080234, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:28:00,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:00,242 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524079854_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:00,242 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20] : 1
2016-05-17 22:28:00,242 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,243 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:00,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:VERTEX_FINISHED]: vertexName=DFE3638F52B4414285A7D1A54F5B8D20, vertexId=vertex_1463524079854_0001_1_01, initRequestedTime=1463524080033, initedTime=1463524080047, startRequestedTime=1463524080062, startedTime=1463524080062, finishTime=1463524080243, timeTaken=181, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080226, Process_Duration=8, Process_End_Time=1463524080234, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1, vertexStats=firstTaskStartTime=1463524080173, firstTasksToStart=[ task_1463524079854_0001_1_01_000000 ], lastTaskFinishTime=1463524080240, lastTasksToFinish=[ task_1463524079854_0001_1_01_000000 ], minTaskDuration=67, maxTaskDuration=67, avgTaskDuration=67.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524079854_0001_1_01_000000 ], longestDurationTasks=[ task_1463524079854_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:00,244 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:00,244 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:00,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:00,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524079854_0001_00_000002, stoppedTime=1463524080244, exitStatus=0
2016-05-17 22:28:00,245 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:00,245 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:00,245 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524079854_0001_1
2016-05-17 22:28:00,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524079854_0001_1
2016-05-17 22:28:00,246 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A5AE188C0C77405C80DC11476990E811 for vertex: vertex_1463524079854_0001_1_01 [DFE3638F52B4414285A7D1A54F5B8D20]
2016-05-17 22:28:00,251 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524079854_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 22:28:00,251 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524079854_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:28:00,251 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524079854_0001_00_000002, Exiting
2016-05-17 22:28:00,251 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524079854_0001_00_000002
2016-05-17 22:28:00,252 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524079854_0001_00_000002
2016-05-17 22:28:00,252 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524079854_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:28:00,254 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524079854_0001_00_000002 completed successfully
2016-05-17 22:28:00,257 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463524079854_0001_1_00 [E41DBFCEB8FC4068BDAEA63828ACADDF]
2016-05-17 22:28:00,259 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524079854_0001_1, queueSize=0
2016-05-17 22:28:00,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:28:00,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524079854_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524079854_0001_1, startTime=1463524080030, finishTime=1463524080257, timeTaken=227, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2891, FILE_BYTES_WRITTEN=704, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=14, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1027604480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=8, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927048160363, Process_Duration=12, Process_End_Time=2927048160375, Read_Duration=2, Tuples_Read=17, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=1
2016-05-17 22:28:00,260 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524079854_0001_1 finished with state: SUCCEEDED
2016-05-17 22:28:00,260 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524079854_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:28:00,260 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524079854_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:28:00 Completed Dag: dag_1463524079854_0001_1
2016-05-17 22:28:00,260 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524079854_0001_1, dagState=SUCCEEDED
2016-05-17 22:28:00,260 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524079854_0001_1_post
2016-05-17 22:28:00,260 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:00,260 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:28:00,260 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524079854_0001_00_000002
2016-05-17 22:28:00,269 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...sRemoval/group/NONE/VALID, applicationId=application_1463524079854_0001
2016-05-17 22:28:00,271 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:28:00,271 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:28:00,272 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:28:00,272 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:28:00,272 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:28:00,277 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:28:00,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:28:00,278 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:28:00,278 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:28:00,279 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:28:00,279 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:28:00,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:28:00,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:28:00,283 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:28:00,283 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:28:00,287 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/4B2DE3EF6C3C41A9BDA8433D9B82F357/EA623E70669041F7B63ACE9B86D984AF/.tez/application_1463524079854_0001
2016-05-17 22:28:00,287 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/VALID/_temporary
2016-05-17 22:28:00,288 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.453
2016-05-17 22:28:00,349 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:28:00,350 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:28:00,392 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 22:28:00,407 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-05-17 22:28:00,407 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:00,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:00,408 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:28:00,425 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:28:00,434 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/VALID
2016-05-17 22:28:00,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:28:00,491 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:00,491 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/VALID"]
2016-05-17 22:28:00,491 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:28:00,491 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:28:00,491 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:28:00,492 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...Removal/group/VALID/VALID
2016-05-17 22:28:00,492 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:28:00,504 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:28:00,508 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:00,509 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:28:00,509 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:28:00,518 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3/.tez/application_1463524080508_0001 doesn't exist and is created
2016-05-17 22:28:00,534 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:28:00,535 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:28:00,535 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3/.tez/application_1463524080508_0001
2016-05-17 22:28:00,539 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524080508_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:00,550 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:28:00,550 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:28:00,552 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:28:00,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:28:00,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3/.tez/application_1463524080508_0001/localmode-log-dir
2016-05-17 22:28:00,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3/.tez/application_1463524080508_0001/localmode-log-dir/history.txt.appattempt_1463524080508_0001_000000, maxErrors=10
2016-05-17 22:28:00,553 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:28:00,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524080508_0001_000000, appSubmitTime=1463524080538, launchTime=1463524080538
2016-05-17 22:28:00,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:28:00,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:28:00,560 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:28:00,561 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:28:00,562 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35658
2016-05-17 22:28:00,565 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:28:00,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524080508_0001_000000, startTime=1463524080567
2016-05-17 22:28:00,567 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:28:00,567 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:28:00,635 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:28:00,636 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:28:00,636 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...Removal/group/VALID/VALID, applicationId=application_1463524080508_0001, dagName=[AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3] test/(1/1) ...Removal/group/VALID/VALID
2016-05-17 22:28:00,652 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3] test/(1/1) ...Removal/group/VALID/VALID
2016-05-17 22:28:00,653 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524080508_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3/.tez/application_1463524080508_0001/localmode-log-dir/dag_1463524080508_0001_1.dot
2016-05-17 22:28:00,654 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3/.tez/application_1463524080508_0001/localmode-log-dir/dag_1463524080508_0001_1-tez-dag.pb.txt
2016-05-17 22:28:00,662 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524080508_0001_1
2016-05-17 22:28:00,662 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:00,662 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3] test/(1/1) ...Removal/group/VALID/VALID
2016-05-17 22:28:00 Running Dag: dag_1463524080508_0001_1
2016-05-17 22:28:00,675 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:28:00,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524080508_0001_1, submitTime=1463524080652
2016-05-17 22:28:00,676 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AACD35286139413D9BD401724BC43C13
2016-05-17 22:28:00,676 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:28:00,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524080508_0001_1, initTime=1463524080675
2016-05-17 22:28:00,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:28:00,678 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524080508_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:28:00,678 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:28:00,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:DAG_STARTED]: dagID=dag_1463524080508_0001_1, startTime=1463524080678
2016-05-17 22:28:00,679 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:28:00,679 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524080508_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:28:00,679 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FDB032FB27A54FCAB11EF540BFE39A7F : {6DA85FECEC7B48EDBA02EFEEE1F1F690={InputName=6DA85FECEC7B48EDBA02EFEEE1F1F690}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:28:00,679 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6DA85FECEC7B48EDBA02EFEEE1F1F690, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:28:00,679 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:00,679 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:00,679 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:00,680 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]. Starting root input initializers: 1
2016-05-17 22:28:00,680 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...Removal/group/VALID/VALID, applicationId=application_1463524080508_0001, dagName=[AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3] test/(1/1) ...Removal/group/VALID/VALID
2016-05-17 22:28:00,682 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:28:00,682 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:00,683 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6DA85FECEC7B48EDBA02EFEEE1F1F690 on vertex vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:00,685 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:28:00,685 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:28:00,686 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:00,686 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:00,686 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:00,686 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B8269F2DF5FE4001AE39AFCCC27C76C0, vertexId=vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:28:00,687 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B8269F2DF5FE4001AE39AFCCC27C76C0, vertexId=vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:00,686 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6DA85FECEC7B48EDBA02EFEEE1F1F690 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:28:00,689 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:28:00,690 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:28:00,690 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B8269F2DF5FE4001AE39AFCCC27C76C0, vertexName=AACD35286139413D9BD401724BC43C13, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:28:00,690 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B8269F2DF5FE4001AE39AFCCC27C76C0, vertexId=vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:00,692 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:28:00,692 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6DA85FECEC7B48EDBA02EFEEE1F1F690 on vertex vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:00,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:00,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AACD35286139413D9BD401724BC43C13, vertexId=vertex_1463524080508_0001_1_01, initRequestedTime=1463524080682, initedTime=1463524080694, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:28:00,694 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:28:00,695 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:00,695 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6DA85FECEC7B48EDBA02EFEEE1F1F690=forAllWorkUnits=true, update=[1]}
2016-05-17 22:28:00,696 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] parallelism set to 1
2016-05-17 22:28:00,696 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:00,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524080508_0001_1_00, eventCount=1
2016-05-17 22:28:00,697 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:00,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FDB032FB27A54FCAB11EF540BFE39A7F, vertexId=vertex_1463524080508_0001_1_00, initRequestedTime=1463524080679, initedTime=1463524080697, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:28:00,698 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] with distanceFromRoot: 0
2016-05-17 22:28:00,698 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:28:00,699 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524080508_0001, with dag id: dag_1463524080508_0001_1
2016-05-17 22:28:00,699 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:00,700 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:00,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524080508_0001_1_00, startRequestedTime=1463524080699, startedTime=1463524080699
2016-05-17 22:28:00,700 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:00,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:TASK_STARTED]: vertexName=FDB032FB27A54FCAB11EF540BFE39A7F, taskId=task_1463524080508_0001_1_00_000000, scheduledTime=1463524080701, launchTime=1463524080701
2016-05-17 22:28:00,701 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524080508_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:00,701 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524080508_0001_1_00 for vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] numStartedSources: 1 numSources: 1
2016-05-17 22:28:00,702 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] with distanceFromRoot: 1
2016-05-17 22:28:00,703 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:00,703 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:00,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:00,708 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FDB032FB27A54FCAB11EF540BFE39A7F in vertex: AACD35286139413D9BD401724BC43C13
2016-05-17 22:28:00,709 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AACD35286139413D9BD401724BC43C13 with 1 source tasks and 1 pending tasks
2016-05-17 22:28:00,709 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:00,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524080508_0001_1_01, startRequestedTime=1463524080708, startedTime=1463524080708
2016-05-17 22:28:00,709 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:00,710 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3] test/(1/1) ...Removal/group/VALID/VALID, VertexName: FDB032FB27A54FCAB11EF540BFE39A7F, VertexParallelism: 1, TaskAttemptID:attempt_1463524080508_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6DA85FECEC7B48EDBA02EFEEE1F1F690, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AACD35286139413D9BD401724BC43C13, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:00,710 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524080508_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:00,711 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:00,711 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:28:00,715 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:28:00,715 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:28:00,716 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524080508_0001_00_000001
2016-05-17 22:28:00,716 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:00,719 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524080508_0001_00_000001
2016-05-17 22:28:00,719 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:00,719 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:00,720 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:00,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524080508_0001_00_000001, launchTime=1463524080719
2016-05-17 22:28:00,721 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524080508_0001_00_000001
2016-05-17 22:28:00,721 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524080508_0001_1_00_000000_0] to container: [container_1463524080508_0001_00_000001]
2016-05-17 22:28:00,722 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:00,722 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524080508_0001_00_000001 given task: attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:00,722 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524080508_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:00,722 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524080508_0001_1_00_000000_0] started. Is using containerId: [container_1463524080508_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:28:00,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FDB032FB27A54FCAB11EF540BFE39A7F, taskAttemptId=attempt_1463524080508_0001_1_00_000000_0, startTime=1463524080722, containerId=container_1463524080508_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524080508_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:28:00,722 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524080508_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:00 Starting to run new task attempt: attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:00,723 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524080508_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:00,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:00,723 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:00,723 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:00,723 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:00,723 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:00,724 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:00,724 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:00,724 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:00,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3] test/(1/1) ...Removal/group/VALID/VALID, VertexName: FDB032FB27A54FCAB11EF540BFE39A7F, VertexParallelism: 1, TaskAttemptID:attempt_1463524080508_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6DA85FECEC7B48EDBA02EFEEE1F1F690, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AACD35286139413D9BD401724BC43C13, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:00,724 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:00,725 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:00,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:00,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,727 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524080508_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:28:00,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6DA85FECEC7B48EDBA02EFEEE1F1F690, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:28:00,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AACD35286139413D9BD401724BC43C13, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:28:00,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:00,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AACD35286139413D9BD401724BC43C13
2016-05-17 22:28:00,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:00,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6DA85FECEC7B48EDBA02EFEEE1F1F690
2016-05-17 22:28:00,731 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6DA85FECEC7B48EDBA02EFEEE1F1F690
2016-05-17 22:28:00,732 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:28:00,732 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AACD35286139413D9BD401724BC43C13
2016-05-17 22:28:00,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AACD35286139413D9BD401724BC43C13
2016-05-17 22:28:00,735 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:28:00,735 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6DA85FECEC7B48EDBA02EFEEE1F1F690
2016-05-17 22:28:00,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6DA85FECEC7B48EDBA02EFEEE1F1F690
2016-05-17 22:28:00,736 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:00,746 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FDB032FB27A54FCAB11EF540BFE39A7F, ordinal: 0
2016-05-17 22:28:00,746 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDB032FB27A54FCAB11EF540BFE39A7F, mem on start (mb), free: 61, total: 490, max: 672
2016-05-17 22:28:00,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:00,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:00,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:00,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:00,747 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:00,747 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:28:00,747 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:00,748 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:28:00,748 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:00,748 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:28:00,749 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6DA85FECEC7B48EDBA02EFEEE1F1F690, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:28:00,749 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AACD35286139413D9BD401724BC43C13, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:28:00,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:00,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6DA85FECEC7B48EDBA02EFEEE1F1F690 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:00,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6DA85FECEC7B48EDBA02EFEEE1F1F690
2016-05-17 22:28:00,750 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:00,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:00,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:00,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6DA85FECEC7B48EDBA02EFEEE1F1F690
2016-05-17 22:28:00,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:00,751 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:00,751 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:28:00,752 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3db14203
2016-05-17 22:28:00,753 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3db14203
2016-05-17 22:28:00,753 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:28:00,753 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:28:00,755 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:00,756 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 6DA85FECEC7B48EDBA02EFEEE1F1F690
2016-05-17 22:28:00,756 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: CC84D80623EF4F49AE335EF59CE77B61
2016-05-17 22:28:00,757 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] CC84D80623EF4F49AE335EF59CE77B61
2016-05-17 22:28:00,759 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@66d2e5d4; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1feef609; comparator=cascading.tuple.hadoop.util.TupleComparator@15ad940; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:28:00,759 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:28:00,760 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:28:00,760 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:00,996 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:28:00,997 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:28:00,997 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:28:00,997 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:28:00,997 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:28:00,998 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:00,998 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:00,998 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FDB032FB27A54FCAB11EF540BFE39A7F, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:28:01,001 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:01,001 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDB032FB27A54FCAB11EF540BFE39A7F, mem on close (mb), free: 244, total: 366, max: 672
2016-05-17 22:28:01,001 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:28:01,001 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:28:01,002 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:28:01,002 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:28:01,004 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:28:01 Completed running task attempt: attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:01,008 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524080508_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:01,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524080508_0001_1_00, eventCount=1
2016-05-17 22:28:01,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FDB032FB27A54FCAB11EF540BFE39A7F, taskAttemptId=attempt_1463524080508_0001_1_00_000000_0, startTime=1463524080722, finishTime=1463524081009, timeTaken=287, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=223, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=384303104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080998, Process_Duration=3, Process_End_Time=1463524081001, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:01,009 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524080508_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:01,009 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524080508_0001_1_00_000000_0
2016-05-17 22:28:01,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:TASK_FINISHED]: vertexName=FDB032FB27A54FCAB11EF540BFE39A7F, taskId=task_1463524080508_0001_1_00_000000, startTime=1463524080722, finishTime=1463524081010, timeTaken=288, status=SUCCEEDED, successfulAttemptID=attempt_1463524080508_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=223, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=384303104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080998, Process_Duration=3, Process_End_Time=1463524081001, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:01,010 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524080508_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:01,010 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] : 1
2016-05-17 22:28:01,011 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:01,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:01,011 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:01,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:01,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:01,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:VERTEX_FINISHED]: vertexName=FDB032FB27A54FCAB11EF540BFE39A7F, vertexId=vertex_1463524080508_0001_1_00, initRequestedTime=1463524080679, initedTime=1463524080697, startRequestedTime=1463524080699, startedTime=1463524080699, finishTime=1463524081011, timeTaken=312, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=223, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=384303104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080998, Process_Duration=3, Process_End_Time=1463524081001, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524080722, firstTasksToStart=[ task_1463524080508_0001_1_00_000000 ], lastTaskFinishTime=1463524081009, lastTasksToFinish=[ task_1463524080508_0001_1_00_000000 ], minTaskDuration=287, maxTaskDuration=287, avgTaskDuration=287.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524080508_0001_1_00_000000 ], longestDurationTasks=[ task_1463524080508_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:01,014 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:01,014 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] attempt: attempt_1463524080508_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:28:01,014 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AACD35286139413D9BD401724BC43C13
2016-05-17 22:28:01,014 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:01,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=223, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=384303104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524080998, Process_Duration=3, Process_End_Time=1463524081001, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:28:01,016 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:01,017 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:01,018 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524080508_0001_00_000001
2016-05-17 22:28:01,018 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:01,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524080508_0001_00_000001, stoppedTime=1463524081018, exitStatus=0
2016-05-17 22:28:01,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:01,019 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:01,019 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:01,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:TASK_STARTED]: vertexName=AACD35286139413D9BD401724BC43C13, taskId=task_1463524080508_0001_1_01_000000, scheduledTime=1463524081019, launchTime=1463524081019
2016-05-17 22:28:01,020 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524080508_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:01,020 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524080508_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:28:01,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:01,021 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:01,021 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:28:01,021 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3] test/(1/1) ...Removal/group/VALID/VALID, VertexName: AACD35286139413D9BD401724BC43C13, VertexParallelism: 1, TaskAttemptID:attempt_1463524080508_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDB032FB27A54FCAB11EF540BFE39A7F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B8269F2DF5FE4001AE39AFCCC27C76C0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:01,021 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524080508_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:01,021 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:01,022 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524080508_0001_00_000002
2016-05-17 22:28:01,022 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:01,023 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(221)) - No more tasks for container with id : container_1463524080508_0001_00_000001. Asking it to die
2016-05-17 22:28:01,023 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524080508_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:28:01,023 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524080508_0001_00_000001, Exiting
2016-05-17 22:28:01,023 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524080508_0001_00_000001
2016-05-17 22:28:01,023 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524080508_0001_00_000001
2016-05-17 22:28:01,026 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524080508_0001_00_000002
2016-05-17 22:28:01,026 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524080508_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:28:01,027 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524080508_0001_00_000001 completed successfully
2016-05-17 22:28:01,027 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:28:01,027 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 22:28:01,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:01,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524080508_0001_00_000002, launchTime=1463524081029
2016-05-17 22:28:01,030 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:01,031 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524080508_0001_00_000002
2016-05-17 22:28:01,032 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524080508_0001_1_01_000000_0] to container: [container_1463524080508_0001_00_000002]
2016-05-17 22:28:01,032 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:01,032 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524080508_0001_00_000002 given task: attempt_1463524080508_0001_1_01_000000_0
2016-05-17 22:28:01,032 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524080508_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524080508_0001_1_01_000000_0
2016-05-17 22:28:01,033 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524080508_0001_1_01_000000_0] started. Is using containerId: [container_1463524080508_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:28:01,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:01,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AACD35286139413D9BD401724BC43C13, taskAttemptId=attempt_1463524080508_0001_1_01_000000_0, startTime=1463524081033, containerId=container_1463524080508_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524080508_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:28:01,034 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524080508_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:01,034 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524080508_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:01 Starting to run new task attempt: attempt_1463524080508_0001_1_01_000000_0
2016-05-17 22:28:01,035 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524080508_0001_1_01_000000_0
2016-05-17 22:28:01,035 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:01,035 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:01,036 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:01,036 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:01,036 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:01,036 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:01,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3] test/(1/1) ...Removal/group/VALID/VALID, VertexName: AACD35286139413D9BD401724BC43C13, VertexParallelism: 1, TaskAttemptID:attempt_1463524080508_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDB032FB27A54FCAB11EF540BFE39A7F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B8269F2DF5FE4001AE39AFCCC27C76C0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:01,037 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:01,037 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524080508_0001_1_01_000000_0
2016-05-17 22:28:01,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:01,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FDB032FB27A54FCAB11EF540BFE39A7F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:28:01,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:01,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B8269F2DF5FE4001AE39AFCCC27C76C0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:28:01,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:01,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B8269F2DF5FE4001AE39AFCCC27C76C0
2016-05-17 22:28:01,043 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:28:01,043 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B8269F2DF5FE4001AE39AFCCC27C76C0
2016-05-17 22:28:01,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,044 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524080508_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:28:01,043 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:01,053 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:28:01,054 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:28:01,054 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,062 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AACD35286139413D9BD401724BC43C13, ordinal: 1
2016-05-17 22:28:01,062 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AACD35286139413D9BD401724BC43C13, mem on start (mb), free: 238, total: 366, max: 672
2016-05-17 22:28:01,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:01,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:01,068 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:28:01,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B8269F2DF5FE4001AE39AFCCC27C76C0
2016-05-17 22:28:01,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:01,068 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:01,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:28:01,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:01,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:28:01,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:01,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:28:01,070 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B8269F2DF5FE4001AE39AFCCC27C76C0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:28:01,070 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FDB032FB27A54FCAB11EF540BFE39A7F, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:28:01,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:01,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FDB032FB27A54FCAB11EF540BFE39A7F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:01,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:01,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:01,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,073 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:01,073 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:28:01,073 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FDB032FB27A54FCAB11EF540BFE39A7F with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:28:01,073 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:28:01,074 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:28:01,074 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:28:01,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:01,079 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524080508_0001_1_01_000000_0
2016-05-17 22:28:01,083 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:28:01,083 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524080508_0001_1_00_000000_0_10057, runDuration: 276230, ]
2016-05-17 22:28:01,084 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: CC84D80623EF4F49AE335EF59CE77B61
2016-05-17 22:28:01,084 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/VALID"], id: B8269F2DF5FE4001AE39AFCCC27C76C0
2016-05-17 22:28:01,084 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/VALID"]
2016-05-17 22:28:01,085 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] CC84D80623EF4F49AE335EF59CE77B61, for 1 inputs
2016-05-17 22:28:01,086 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:28:01,086 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:28:01,090 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524080508_0001_1_00_000000_0_10057] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524081090, TimeTaken=4, Rate=0.16 MB/s
2016-05-17 22:28:01,091 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,091 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:28:01,092 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [FDB032FB27A54FCAB11EF540BFE39A7F] #10 in 6ms
2016-05-17 22:28:01,092 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FDB032FB27A54FCAB11EF540BFE39A7F] #10, status:false, isInterrupted:false
2016-05-17 22:28:01,093 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:28:01,094 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:28:01,094 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:01,094 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:28:01,095 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:28:01,095 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,095 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:28:01,096 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AACD35286139413D9BD401724BC43C13, all 1 inputs ready in: 00:00:00.010
2016-05-17 22:28:01,097 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7267c453; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7a8a5fd2
2016-05-17 22:28:01,099 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:01,100 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:01,101 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:01,103 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:28:01,103 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:28:01,104 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524080508_0001_1_01_000000_0
2016-05-17 22:28:01,104 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524080508_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:28:01,105 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524080508_0001_1_01_000000_0
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:01,108 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=384303104, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524081096, Process_Duration=9, Process_End_Time=1463524081105, Read_Duration=1, Tuples_Read=7]]
2016-05-17 22:28:01,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AACD35286139413D9BD401724BC43C13, taskAttemptId=attempt_1463524080508_0001_1_01_000000_0, startTime=1463524081033, finishTime=1463524081108, timeTaken=75, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=384303104, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524081096, Process_Duration=9, Process_End_Time=1463524081105, Read_Duration=1, Tuples_Read=7
2016-05-17 22:28:01,110 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524080508_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:28:01,110 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524080508_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:28:01,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:TASK_FINISHED]: vertexName=AACD35286139413D9BD401724BC43C13, taskId=task_1463524080508_0001_1_01_000000, startTime=1463524081033, finishTime=1463524081110, timeTaken=77, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-17 22:28:01,110 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524080508_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:28:01,111 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] : 1
2016-05-17 22:28:01,111 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] because task failed: task_1463524080508_0001_1_01_000000
2016-05-17 22:28:01,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:01,111 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:01,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:01,111 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:28:01,113 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:28:01,113 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:01,114 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=B8269F2DF5FE4001AE39AFCCC27C76C0, vertexId=vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:01,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:VERTEX_FINISHED]: vertexName=AACD35286139413D9BD401724BC43C13, vertexId=vertex_1463524080508_0001_1_01, initRequestedTime=1463524080682, initedTime=1463524080694, startRequestedTime=1463524080708, startedTime=1463524080708, finishTime=1463524081115, timeTaken=407, status=FAILED, diagnostics=Task failed, taskId=task_1463524080508_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524081033, firstTasksToStart=[ task_1463524080508_0001_1_01_000000 ], lastTaskFinishTime=1463524081108, lastTasksToFinish=[ task_1463524080508_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:28:01,116 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:28:01,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524080508_0001_00_000002, stoppedTime=1463524081116, exitStatus=0
2016-05-17 22:28:01,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:01,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:01,116 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:01,118 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524080508_0001_1_01_000000_0 failed: true container: container_1463524080508_0001_00_000002 numFailedTAs: 0
2016-05-17 22:28:01,119 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 22:28:01,119 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:28:01,119 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524080508_0001_1
2016-05-17 22:28:01,119 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463524080508_0001_1_00 [FDB032FB27A54FCAB11EF540BFE39A7F]
2016-05-17 22:28:01,119 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: B8269F2DF5FE4001AE39AFCCC27C76C0 for vertex: vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13]
2016-05-17 22:28:01,122 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524080508_0001_1, queueSize=0
2016-05-17 22:28:01,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:28:01,109 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AACD35286139413D9BD401724BC43C13, mem on close (mb), free: 234, total: 366, max: 672
2016-05-17 22:28:01,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=AACD35286139413D9BD401724BC43C13, index=1
2016-05-17 22:28:01,122 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,128 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:28:01,129 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:28:01,129 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:01,129 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:28:01,129 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:28:01,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=AACD35286139413D9BD401724BC43C13, sourceVertex=FDB032FB27A54FCAB11EF540BFE39A7F
2016-05-17 22:28:01,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=AACD35286139413D9BD401724BC43C13, sourceVertex=B8269F2DF5FE4001AE39AFCCC27C76C0
2016-05-17 22:28:01,130 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524080508_0001_00_000002
2016-05-17 22:28:01,130 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524080508_0001_00_000002
2016-05-17 22:28:01,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524080508_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524080508_0001_1, startTime=1463524080678, finishTime=1463524081120, timeTaken=442, status=FAILED, diagnostics=Vertex failed, vertexName=AACD35286139413D9BD401724BC43C13, vertexId=vertex_1463524080508_0001_1_01, diagnostics=[Task failed, taskId=task_1463524080508_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.runGroupAssertions(AppliedAssertionsPlatformTest.java:283)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524080508_0001_1_01 [AACD35286139413D9BD401724BC43C13] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=223, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=384303104, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524080998, Process_Duration=3, Process_End_Time=1463524081001, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:01,131 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524080508_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:28:01,131 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524080508_0001_00_000002 completed but with errors
2016-05-17 22:28:01,131 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524080508_0001_1 finished with state: FAILED
2016-05-17 22:28:01,133 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524080508_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:28:01,134 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524080508_0001_00_000002
2016-05-17 22:28:01 Completed Dag: dag_1463524080508_0001_1
2016-05-17 22:28:01,134 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524080508_0001_1, dagState=FAILED
2016-05-17 22:28:01,134 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524080508_0001_1_post
2016-05-17 22:28:01,134 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:01,135 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:28:01,135 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524080508_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:28:01,135 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524080508_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:28:01,135 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524080508_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:28:01,139 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...Removal/group/VALID/VALID, applicationId=application_1463524080508_0001
2016-05-17 22:28:01,146 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:28:01,146 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:28:01,146 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:28:01,146 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:28:01,146 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:28:01,147 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:28:01,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:28:01,150 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:28:01,150 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:28:01,150 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:28:01,150 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:28:01,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:28:01,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:28:01,156 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/AC5FCB7517BE4A6696272870D762E2CA/BB55DD26AC2C4D6E94BC98DC88530DF3/.tez/application_1463524080508_0001
2016-05-17 22:28:01,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping all jobs
2016-05-17 22:28:01,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping: (1/1) ...Removal/group/VALID/VALID
2016-05-17 22:28:01,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopped all jobs
2016-05-17 22:28:01,158 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/VALID/_temporary
2016-05-17 22:28:01,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.669
2016-05-17 22:28:01,159 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/STRICT
2016-05-17 22:28:01,159 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/STRICT/STRICT
2016-05-17 22:28:01,160 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/STRICT
2016-05-17 22:28:01,160 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/VALID/VALID
2016-05-17 22:28:01,160 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testGroupAssertionsRemoval/group/NONE/VALID
2016-05-17 22:28:01,201 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:28:01,202 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:28:01,254 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-05-17 22:28:01,271 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.037
2016-05-17 22:28:01,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:01,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:01,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:28:01,288 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:28:01,296 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsFail/fail
2016-05-17 22:28:01,352 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:28:01,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:01,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsFail/fail"]
2016-05-17 22:28:01,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:28:01,353 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:28:01,354 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:28:01,354 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tGroupAssertionsFail/fail
2016-05-17 22:28:01,355 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:28:01,372 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:28:01,377 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:01,378 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:28:01,378 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:28:01,388 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA/.tez/application_1463524081378_0001 doesn't exist and is created
2016-05-17 22:28:01,407 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:28:01,407 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:28:01,407 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA/.tez/application_1463524081378_0001
2016-05-17 22:28:01,411 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524081378_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:01,421 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:28:01,422 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:28:01,423 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:28:01,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:28:01,423 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA/.tez/application_1463524081378_0001/localmode-log-dir
2016-05-17 22:28:01,424 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA/.tez/application_1463524081378_0001/localmode-log-dir/history.txt.appattempt_1463524081378_0001_000000, maxErrors=10
2016-05-17 22:28:01,424 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:28:01,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524081378_0001_000000, appSubmitTime=1463524081410, launchTime=1463524081410
2016-05-17 22:28:01,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:28:01,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:28:01,429 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:28:01,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:28:01,432 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58971
2016-05-17 22:28:01,433 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:28:01,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524081378_0001_000000, startTime=1463524081435
2016-05-17 22:28:01,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:28:01,435 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:28:01,507 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:28:01,507 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:28:01,508 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tGroupAssertionsFail/fail, applicationId=application_1463524081378_0001, dagName=[3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA] test/(1/1) ...tGroupAssertionsFail/fail
2016-05-17 22:28:01,523 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA] test/(1/1) ...tGroupAssertionsFail/fail
2016-05-17 22:28:01,524 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524081378_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA/.tez/application_1463524081378_0001/localmode-log-dir/dag_1463524081378_0001_1.dot
2016-05-17 22:28:01,525 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA/.tez/application_1463524081378_0001/localmode-log-dir/dag_1463524081378_0001_1-tez-dag.pb.txt
2016-05-17 22:28:01,531 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524081378_0001_1
2016-05-17 22:28:01,532 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:01,532 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA] test/(1/1) ...tGroupAssertionsFail/fail
2016-05-17 22:28:01 Running Dag: dag_1463524081378_0001_1
2016-05-17 22:28:01,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524081378_0001_1, submitTime=1463524081523
2016-05-17 22:28:01,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:28:01,545 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F8A5BE79B92347F3B1DF46FB6BCB1C14
2016-05-17 22:28:01,545 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:28:01,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524081378_0001_1, initTime=1463524081544
2016-05-17 22:28:01,545 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:28:01,546 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524081378_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:28:01,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:DAG_STARTED]: dagID=dag_1463524081378_0001_1, startTime=1463524081546
2016-05-17 22:28:01,547 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:28:01,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:28:01,547 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524081378_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:28:01,548 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F11F997066AC4B9FA8ECDFDF026621DD : {8D48705E7B584B3782E32687D6541559={InputName=8D48705E7B584B3782E32687D6541559}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:28:01,548 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8D48705E7B584B3782E32687D6541559, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:28:01,548 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,548 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tGroupAssertionsFail/fail, applicationId=application_1463524081378_0001, dagName=[3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA] test/(1/1) ...tGroupAssertionsFail/fail
2016-05-17 22:28:01,548 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,548 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,549 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]. Starting root input initializers: 1
2016-05-17 22:28:01,549 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:28:01,549 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,552 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:28:01,552 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,552 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,553 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,553 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=453817E1510741B38C22D630B364F853, vertexId=vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:28:01,553 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=453817E1510741B38C22D630B364F853, vertexId=vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,553 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8D48705E7B584B3782E32687D6541559 on vertex vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,555 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=453817E1510741B38C22D630B364F853, vertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:28:01,555 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=453817E1510741B38C22D630B364F853, vertexId=vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,556 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:28:01,556 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8D48705E7B584B3782E32687D6541559 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:28:01,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:01,557 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:28:01,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, vertexId=vertex_1463524081378_0001_1_01, initRequestedTime=1463524081549, initedTime=1463524081556, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:28:01,557 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:28:01,557 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:28:01,559 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:28:01,560 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8D48705E7B584B3782E32687D6541559 on vertex vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,560 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:01,560 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8D48705E7B584B3782E32687D6541559=forAllWorkUnits=true, update=[1]}
2016-05-17 22:28:01,560 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] parallelism set to 1
2016-05-17 22:28:01,561 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:01,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524081378_0001_1_00, eventCount=1
2016-05-17 22:28:01,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:01,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F11F997066AC4B9FA8ECDFDF026621DD, vertexId=vertex_1463524081378_0001_1_00, initRequestedTime=1463524081548, initedTime=1463524081561, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:28:01,562 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] with distanceFromRoot: 0
2016-05-17 22:28:01,562 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:28:01,563 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,563 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:01,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524081378_0001_1_00, startRequestedTime=1463524081563, startedTime=1463524081563
2016-05-17 22:28:01,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524081378_0001, with dag id: dag_1463524081378_0001_1
2016-05-17 22:28:01,564 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:01,565 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:01,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:TASK_STARTED]: vertexName=F11F997066AC4B9FA8ECDFDF026621DD, taskId=task_1463524081378_0001_1_00_000000, scheduledTime=1463524081565, launchTime=1463524081565
2016-05-17 22:28:01,566 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524081378_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:01,568 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524081378_0001_1_00 for vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] numStartedSources: 1 numSources: 1
2016-05-17 22:28:01,569 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] with distanceFromRoot: 1
2016-05-17 22:28:01,569 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,569 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F11F997066AC4B9FA8ECDFDF026621DD in vertex: F8A5BE79B92347F3B1DF46FB6BCB1C14
2016-05-17 22:28:01,570 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F8A5BE79B92347F3B1DF46FB6BCB1C14 with 1 source tasks and 1 pending tasks
2016-05-17 22:28:01,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:01,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524081378_0001_1_01, startRequestedTime=1463524081569, startedTime=1463524081569
2016-05-17 22:28:01,570 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:01,571 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA] test/(1/1) ...tGroupAssertionsFail/fail, VertexName: F11F997066AC4B9FA8ECDFDF026621DD, VertexParallelism: 1, TaskAttemptID:attempt_1463524081378_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8D48705E7B584B3782E32687D6541559, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:01,571 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524081378_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:01,571 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:01,572 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:28:01,572 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:28:01,573 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:28:01,573 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524081378_0001_00_000001
2016-05-17 22:28:01,573 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:01,575 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524081378_0001_00_000001
2016-05-17 22:28:01,575 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:01,576 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:01,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:01,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524081378_0001_00_000001, launchTime=1463524081576
2016-05-17 22:28:01,577 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524081378_0001_00_000001
2016-05-17 22:28:01,578 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524081378_0001_1_00_000000_0] to container: [container_1463524081378_0001_00_000001]
2016-05-17 22:28:01,578 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:01,578 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524081378_0001_00_000001 given task: attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01,578 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524081378_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01 Starting to run new task attempt: attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01,578 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524081378_0001_1_00_000000_0] started. Is using containerId: [container_1463524081378_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:28:01,578 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F11F997066AC4B9FA8ECDFDF026621DD, taskAttemptId=attempt_1463524081378_0001_1_00_000000_0, startTime=1463524081578, containerId=container_1463524081378_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524081378_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:28:01,578 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:01,578 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524081378_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:01,579 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:01,579 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524081378_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:01,579 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:01,579 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:01,579 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:01,579 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:01,579 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:01,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA] test/(1/1) ...tGroupAssertionsFail/fail, VertexName: F11F997066AC4B9FA8ECDFDF026621DD, VertexParallelism: 1, TaskAttemptID:attempt_1463524081378_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8D48705E7B584B3782E32687D6541559, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:01,580 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:01,580 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:01,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8D48705E7B584B3782E32687D6541559, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:28:01,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,582 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:28:01,582 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524081378_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:28:01,582 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:01,582 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F8A5BE79B92347F3B1DF46FB6BCB1C14
2016-05-17 22:28:01,584 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:28:01,584 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F8A5BE79B92347F3B1DF46FB6BCB1C14
2016-05-17 22:28:01,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F8A5BE79B92347F3B1DF46FB6BCB1C14
2016-05-17 22:28:01,585 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:01,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:01,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8D48705E7B584B3782E32687D6541559
2016-05-17 22:28:01,587 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8D48705E7B584B3782E32687D6541559
2016-05-17 22:28:01,590 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:28:01,590 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8D48705E7B584B3782E32687D6541559
2016-05-17 22:28:01,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8D48705E7B584B3782E32687D6541559
2016-05-17 22:28:01,593 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F11F997066AC4B9FA8ECDFDF026621DD, ordinal: 0
2016-05-17 22:28:01,594 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F11F997066AC4B9FA8ECDFDF026621DD, mem on start (mb), free: 155, total: 366, max: 672
2016-05-17 22:28:01,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:01,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:01,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:01,594 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:01,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:28:01,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:28:01,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:01,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:01,596 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:28:01,596 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F8A5BE79B92347F3B1DF46FB6BCB1C14, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:28:01,596 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8D48705E7B584B3782E32687D6541559, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:28:01,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:01,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8D48705E7B584B3782E32687D6541559 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:01,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:01,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8D48705E7B584B3782E32687D6541559
2016-05-17 22:28:01,597 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:01,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8D48705E7B584B3782E32687D6541559
2016-05-17 22:28:01,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:01,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:01,598 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01,600 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:28:01,600 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:01,601 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@710e3ef3
2016-05-17 22:28:01,601 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 8D48705E7B584B3782E32687D6541559
2016-05-17 22:28:01,601 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: 808C94C3A2EA4546A06B82EC023DE9B6
2016-05-17 22:28:01,602 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] 808C94C3A2EA4546A06B82EC023DE9B6
2016-05-17 22:28:01,602 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@710e3ef3
2016-05-17 22:28:01,602 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:28:01,602 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:28:01,605 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@7af31547; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6cfa2760; comparator=cascading.tuple.hadoop.util.TupleComparator@1b2f4785; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:28:01,605 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:28:01,605 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:28:01,606 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:01,837 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:28:01,837 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:28:01,841 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:28:01,842 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:28:01,842 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:28:01,842 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:28:01,842 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:28:01,843 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:01,843 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:01,844 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F11F997066AC4B9FA8ECDFDF026621DD, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:28:01,847 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01,847 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F11F997066AC4B9FA8ECDFDF026621DD, mem on close (mb), free: 355, total: 479, max: 672
2016-05-17 22:28:01,847 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:28:01,848 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:28:01,848 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:28:01,848 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:28:01,852 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:28:01 Completed running task attempt: attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01,856 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524081378_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:01,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524081378_0001_1_00, eventCount=1
2016-05-17 22:28:01,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F11F997066AC4B9FA8ECDFDF026621DD, taskAttemptId=attempt_1463524081378_0001_1_00_000000_0, startTime=1463524081578, finishTime=1463524081858, timeTaken=280, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=222, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=502267904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524081844, Process_Duration=3, Process_End_Time=1463524081847, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:01,858 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524081378_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:01,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:01,858 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524081378_0001_1_00_000000_0
2016-05-17 22:28:01,859 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:01,859 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:01,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:TASK_FINISHED]: vertexName=F11F997066AC4B9FA8ECDFDF026621DD, taskId=task_1463524081378_0001_1_00_000000, startTime=1463524081578, finishTime=1463524081858, timeTaken=280, status=SUCCEEDED, successfulAttemptID=attempt_1463524081378_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=222, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=502267904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524081844, Process_Duration=3, Process_End_Time=1463524081847, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:01,859 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524081378_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:01,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=222, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=502267904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524081844, Process_Duration=3, Process_End_Time=1463524081847, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:28:01,861 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:01,861 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:01,861 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524081378_0001_00_000001
2016-05-17 22:28:01,861 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] : 1
2016-05-17 22:28:01,861 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,863 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:01,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:VERTEX_FINISHED]: vertexName=F11F997066AC4B9FA8ECDFDF026621DD, vertexId=vertex_1463524081378_0001_1_00, initRequestedTime=1463524081548, initedTime=1463524081561, startRequestedTime=1463524081563, startedTime=1463524081563, finishTime=1463524081861, timeTaken=298, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=222, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=502267904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524081844, Process_Duration=3, Process_End_Time=1463524081847, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524081578, firstTasksToStart=[ task_1463524081378_0001_1_00_000000 ], lastTaskFinishTime=1463524081858, lastTasksToFinish=[ task_1463524081378_0001_1_00_000000 ], minTaskDuration=280, maxTaskDuration=280, avgTaskDuration=280.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524081378_0001_1_00_000000 ], longestDurationTasks=[ task_1463524081378_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:01,863 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:01,863 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:01,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524081378_0001_00_000001, stoppedTime=1463524081864, exitStatus=0
2016-05-17 22:28:01,864 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:01,864 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] attempt: attempt_1463524081378_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:28:01,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:01,864 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F8A5BE79B92347F3B1DF46FB6BCB1C14
2016-05-17 22:28:01,864 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,865 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:01,865 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524081378_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:28:01,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:TASK_STARTED]: vertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, taskId=task_1463524081378_0001_1_01_000000, scheduledTime=1463524081865, launchTime=1463524081865
2016-05-17 22:28:01,865 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524081378_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:01,865 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:28:01,865 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA] test/(1/1) ...tGroupAssertionsFail/fail, VertexName: F8A5BE79B92347F3B1DF46FB6BCB1C14, VertexParallelism: 1, TaskAttemptID:attempt_1463524081378_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F11F997066AC4B9FA8ECDFDF026621DD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=453817E1510741B38C22D630B364F853, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:01,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:01,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524081378_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:01,866 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:01,867 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524081378_0001_00_000002
2016-05-17 22:28:01,867 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:01,868 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524081378_0001_00_000002
2016-05-17 22:28:01,868 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:01,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:01,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:01,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524081378_0001_00_000002, launchTime=1463524081869
2016-05-17 22:28:01,870 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524081378_0001_00_000002
2016-05-17 22:28:01,870 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524081378_0001_1_01_000000_0] to container: [container_1463524081378_0001_00_000002]
2016-05-17 22:28:01,871 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:01,871 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524081378_0001_00_000002 given task: attempt_1463524081378_0001_1_01_000000_0
2016-05-17 22:28:01,871 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524081378_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524081378_0001_1_01_000000_0
2016-05-17 22:28:01,871 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524081378_0001_1_01_000000_0] started. Is using containerId: [container_1463524081378_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:28:01,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, taskAttemptId=attempt_1463524081378_0001_1_01_000000_0, startTime=1463524081871, containerId=container_1463524081378_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524081378_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:28:01,871 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524081378_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:28:01,872 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524081378_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:01 Starting to run new task attempt: attempt_1463524081378_0001_1_01_000000_0
2016-05-17 22:28:01,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:01,872 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524081378_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:28:01,872 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524081378_0001_00_000001, Exiting
2016-05-17 22:28:01,872 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524081378_0001_00_000001
2016-05-17 22:28:01,873 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524081378_0001_00_000001
2016-05-17 22:28:01,872 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524081378_0001_1_01_000000_0
2016-05-17 22:28:01,872 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524081378_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:01,875 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:01,875 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:01,875 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:01,875 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:01,875 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:01,875 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:01,876 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA] test/(1/1) ...tGroupAssertionsFail/fail, VertexName: F8A5BE79B92347F3B1DF46FB6BCB1C14, VertexParallelism: 1, TaskAttemptID:attempt_1463524081378_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F11F997066AC4B9FA8ECDFDF026621DD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=453817E1510741B38C22D630B364F853, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:01,876 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:01,874 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524081378_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:28:01,876 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524081378_0001_00_000001 completed successfully
2016-05-17 22:28:01,877 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:28:01,878 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524081378_0001_1_01_000000_0
2016-05-17 22:28:01,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:01,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F11F997066AC4B9FA8ECDFDF026621DD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:28:01,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:01,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,881 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:28:01,881 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:28:01,881 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,882 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:01,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=453817E1510741B38C22D630B364F853, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:28:01,889 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524081378_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:28:01,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:01,891 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 453817E1510741B38C22D630B364F853
2016-05-17 22:28:01,892 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:28:01,892 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 453817E1510741B38C22D630B364F853
2016-05-17 22:28:01,900 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F8A5BE79B92347F3B1DF46FB6BCB1C14, ordinal: 1
2016-05-17 22:28:01,900 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8A5BE79B92347F3B1DF46FB6BCB1C14, mem on start (mb), free: 348, total: 479, max: 672
2016-05-17 22:28:01,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:01,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:01,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:01,907 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:28:01,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 453817E1510741B38C22D630B364F853
2016-05-17 22:28:01,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:01,907 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:01,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:28:01,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:28:01,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:01,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:01,910 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:28:01,910 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F11F997066AC4B9FA8ECDFDF026621DD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:28:01,910 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 453817E1510741B38C22D630B364F853, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:28:01,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:01,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F11F997066AC4B9FA8ECDFDF026621DD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:01,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:01,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:01,911 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:01,912 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:28:01,912 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F11F997066AC4B9FA8ECDFDF026621DD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:28:01,912 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:28:01,913 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:28:01,913 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:28:01,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:01,921 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524081378_0001_1_00_000000_0_10063, runDuration: 272098, ]
2016-05-17 22:28:01,919 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524081378_0001_1_01_000000_0
2016-05-17 22:28:01,923 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524081378_0001_1_00_000000_0_10063] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524081923, TimeTaken=1, Rate=0.62 MB/s
2016-05-17 22:28:01,923 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,923 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:28:01,923 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [F11F997066AC4B9FA8ECDFDF026621DD] #11 in 1ms
2016-05-17 22:28:01,924 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F11F997066AC4B9FA8ECDFDF026621DD] #11, status:false, isInterrupted:false
2016-05-17 22:28:01,925 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:28:01,925 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:28:01,926 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: 808C94C3A2EA4546A06B82EC023DE9B6
2016-05-17 22:28:01,926 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:28:01,926 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsFail/fail"], id: 453817E1510741B38C22D630B364F853
2016-05-17 22:28:01,926 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:01,926 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:28:01,926 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsFail/fail"]
2016-05-17 22:28:01,927 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] 808C94C3A2EA4546A06B82EC023DE9B6, for 1 inputs
2016-05-17 22:28:01,927 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:28:01,930 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,930 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F8A5BE79B92347F3B1DF46FB6BCB1C14, all 1 inputs ready in: 00:00:00.003
2016-05-17 22:28:01,931 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:28:01,932 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@261e733f; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@192040a6
2016-05-17 22:28:01,934 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testGroupAssertionsFail(AppliedAssertionsPlatformTest.java:236)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:01,934 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testGroupAssertionsFail(AppliedAssertionsPlatformTest.java:236)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:01,935 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testGroupAssertionsFail(AppliedAssertionsPlatformTest.java:236)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:01,936 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:28:01,936 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:28:01,936 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524081378_0001_1_01_000000_0
2016-05-17 22:28:01,937 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524081378_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:28:01,937 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524081378_0001_1_01_000000_0
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testGroupAssertionsFail(AppliedAssertionsPlatformTest.java:236)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:01,948 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=502267904, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524081931, Process_Duration=6, Process_End_Time=1463524081937, Read_Duration=1, Tuples_Read=7]]
2016-05-17 22:28:01,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, taskAttemptId=attempt_1463524081378_0001_1_01_000000_0, startTime=1463524081871, finishTime=1463524081948, timeTaken=77, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testGroupAssertionsFail(AppliedAssertionsPlatformTest.java:236)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=502267904, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=18, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524081931, Process_Duration=6, Process_End_Time=1463524081937, Read_Duration=1, Tuples_Read=7
2016-05-17 22:28:01,948 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524081378_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:28:01,949 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524081378_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:28:01,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:01,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:TASK_FINISHED]: vertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, taskId=task_1463524081378_0001_1_01_000000, startTime=1463524081871, finishTime=1463524081949, timeTaken=78, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testGroupAssertionsFail(AppliedAssertionsPlatformTest.java:236)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-17 22:28:01,949 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524081378_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:28:01,949 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:01,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:01,949 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] : 1
2016-05-17 22:28:01,952 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] because task failed: task_1463524081378_0001_1_01_000000
2016-05-17 22:28:01,952 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:28:01,953 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:28:01,953 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,953 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=453817E1510741B38C22D630B364F853, vertexId=vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:VERTEX_FINISHED]: vertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, vertexId=vertex_1463524081378_0001_1_01, initRequestedTime=1463524081549, initedTime=1463524081556, startRequestedTime=1463524081569, startedTime=1463524081569, finishTime=1463524081954, timeTaken=385, status=FAILED, diagnostics=Task failed, taskId=task_1463524081378_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testGroupAssertionsFail(AppliedAssertionsPlatformTest.java:236)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524081871, firstTasksToStart=[ task_1463524081378_0001_1_01_000000 ], lastTaskFinishTime=1463524081948, lastTasksToFinish=[ task_1463524081378_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:28:01,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:01,955 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:28:01,956 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524081378_0001_00_000002, stoppedTime=1463524081956, exitStatus=0
2016-05-17 22:28:01,956 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:01,956 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:01,957 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524081378_0001_1_01_000000_0 failed: true container: container_1463524081378_0001_00_000002 numFailedTAs: 0
2016-05-17 22:28:01,957 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 22:28:01,957 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:28:01,957 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524081378_0001_1
2016-05-17 22:28:01,957 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 453817E1510741B38C22D630B364F853 for vertex: vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14]
2016-05-17 22:28:01,958 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463524081378_0001_1_00 [F11F997066AC4B9FA8ECDFDF026621DD]
2016-05-17 22:28:01,959 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524081378_0001_1, queueSize=0
2016-05-17 22:28:01,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:28:01,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524081378_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524081378_0001_1, startTime=1463524081546, finishTime=1463524081958, timeTaken=412, status=FAILED, diagnostics=Vertex failed, vertexName=F8A5BE79B92347F3B1DF46FB6BCB1C14, vertexId=vertex_1463524081378_0001_1_01, diagnostics=[Task failed, taskId=task_1463524081378_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testGroupAssertionsFail(AppliedAssertionsPlatformTest.java:236)] group size: 7, is not equal to: 0, in group ['method']: ['POST']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertGroupBase.doAssert(AssertGroupBase.java:172)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:127)
at cascading.flow.stream.element.GroupAssertionEveryStage.completeGroup(GroupAssertionEveryStage.java:36)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524081378_0001_1_01 [F8A5BE79B92347F3B1DF46FB6BCB1C14] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=222, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=502267904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524081844, Process_Duration=3, Process_End_Time=1463524081847, Read_Duration=2, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:01,960 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524081378_0001_1 finished with state: FAILED
2016-05-17 22:28:01,960 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524081378_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:28:01,961 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524081378_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 22:28:01 Completed Dag: dag_1463524081378_0001_1
2016-05-17 22:28:01,961 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524081378_0001_1, dagState=FAILED
2016-05-17 22:28:01,961 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524081378_0001_1_post
2016-05-17 22:28:01,961 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:01,961 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:28:01,966 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:28:01,967 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8A5BE79B92347F3B1DF46FB6BCB1C14, mem on close (mb), free: 344, total: 479, max: 672
2016-05-17 22:28:01,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F8A5BE79B92347F3B1DF46FB6BCB1C14, index=1
2016-05-17 22:28:01,967 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,972 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:28:01,973 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:28:01,973 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:01,973 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:28:01,973 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:28:01,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F8A5BE79B92347F3B1DF46FB6BCB1C14, sourceVertex=F11F997066AC4B9FA8ECDFDF026621DD
2016-05-17 22:28:01,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F8A5BE79B92347F3B1DF46FB6BCB1C14, sourceVertex=453817E1510741B38C22D630B364F853
2016-05-17 22:28:01,973 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524081378_0001_00_000002
2016-05-17 22:28:01,973 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524081378_0001_00_000002
2016-05-17 22:28:01,974 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524081378_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:28:01,974 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524081378_0001_00_000002 completed but with errors
2016-05-17 22:28:01,974 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524081378_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:28:01,974 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524081378_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:28:01,974 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524081378_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:28:01,975 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tGroupAssertionsFail/fail, applicationId=application_1463524081378_0001
2016-05-17 22:28:01,980 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:28:01,981 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:28:01,981 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:28:01,981 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:28:01,981 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:28:01,983 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:28:01,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:28:01,984 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:28:01,984 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:28:01,984 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:28:01,984 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:28:01,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:28:01,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:28:01,990 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3CC0FCAA6F9143C9BC9988A1E293037F/EE7A885DC29E4FCDB54BA11C7018DDEA/.tez/application_1463524081378_0001
2016-05-17 22:28:01,991 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping all jobs
2016-05-17 22:28:01,991 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping: (1/1) ...tGroupAssertionsFail/fail
2016-05-17 22:28:01,992 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopped all jobs
2016-05-17 22:28:01,992 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsFail/fail/_temporary
2016-05-17 22:28:01,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.640
2016-05-17 22:28:01,993 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testGroupAssertionsFail/fail
2016-05-17 22:28:02,029 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:28:02,030 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:28:02,082 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 22:28:02,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-17 22:28:02,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:02,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:02,093 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:28:02,108 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:28:02,115 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsPass/pass
2016-05-17 22:28:02,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:28:02,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:02,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsPass/pass"]
2016-05-17 22:28:02,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:28:02,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:28:02,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:28:02,165 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...tGroupAssertionsPass/pass
2016-05-17 22:28:02,165 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:28:02,177 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:28:02,182 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:02,182 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:28:02,182 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:28:02,193 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C/.tez/application_1463524082182_0001 doesn't exist and is created
2016-05-17 22:28:02,210 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:28:02,210 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:28:02,211 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C/.tez/application_1463524082182_0001
2016-05-17 22:28:02,215 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524082182_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:02,225 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:28:02,226 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:28:02,227 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:28:02,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:28:02,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C/.tez/application_1463524082182_0001/localmode-log-dir
2016-05-17 22:28:02,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C/.tez/application_1463524082182_0001/localmode-log-dir/history.txt.appattempt_1463524082182_0001_000000, maxErrors=10
2016-05-17 22:28:02,229 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:28:02,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524082182_0001_000000, appSubmitTime=1463524082214, launchTime=1463524082214
2016-05-17 22:28:02,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:28:02,236 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:28:02,237 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:28:02,242 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46901
2016-05-17 22:28:02,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:28:02,251 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:28:02,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:28:02,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524082182_0001_000000, startTime=1463524082253
2016-05-17 22:28:02,254 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:28:02,311 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:28:02,311 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:28:02,311 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tGroupAssertionsPass/pass, applicationId=application_1463524082182_0001, dagName=[3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C] test/(1/1) ...tGroupAssertionsPass/pass
2016-05-17 22:28:02,327 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C] test/(1/1) ...tGroupAssertionsPass/pass
2016-05-17 22:28:02,328 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524082182_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C/.tez/application_1463524082182_0001/localmode-log-dir/dag_1463524082182_0001_1.dot
2016-05-17 22:28:02,329 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C/.tez/application_1463524082182_0001/localmode-log-dir/dag_1463524082182_0001_1-tez-dag.pb.txt
2016-05-17 22:28:02,336 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524082182_0001_1
2016-05-17 22:28:02,336 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:02,336 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C] test/(1/1) ...tGroupAssertionsPass/pass
2016-05-17 22:28:02 Running Dag: dag_1463524082182_0001_1
2016-05-17 22:28:02,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524082182_0001_1, submitTime=1463524082327
2016-05-17 22:28:02,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:28:02,349 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EF38E69CA5A64637B85AD5AA61F789EF
2016-05-17 22:28:02,349 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:28:02,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524082182_0001_1, initTime=1463524082348
2016-05-17 22:28:02,350 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524082182_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:28:02,349 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:28:02,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:DAG_STARTED]: dagID=dag_1463524082182_0001_1, startTime=1463524082350
2016-05-17 22:28:02,350 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:28:02,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:28:02,350 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524082182_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:28:02,351 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D6FB59917DDD419D85F83818C013C301 : {CCAFF66EB8A84A40B2824E412BE2285B={InputName=CCAFF66EB8A84A40B2824E412BE2285B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:28:02,351 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: CCAFF66EB8A84A40B2824E412BE2285B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:28:02,351 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,351 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,351 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,352 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]. Starting root input initializers: 1
2016-05-17 22:28:02,352 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:28:02,352 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: CCAFF66EB8A84A40B2824E412BE2285B on vertex vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,352 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tGroupAssertionsPass/pass, applicationId=application_1463524082182_0001, dagName=[3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C] test/(1/1) ...tGroupAssertionsPass/pass
2016-05-17 22:28:02,353 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:28:02,354 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input CCAFF66EB8A84A40B2824E412BE2285B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:28:02,354 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:28:02,354 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:28:02,355 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:28:02,355 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: CCAFF66EB8A84A40B2824E412BE2285B on vertex vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,352 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,359 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:28:02,359 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,359 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,359 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,359 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3DBAD1EEB9D545F7B29C868C5FAEBBE9, vertexId=vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:28:02,359 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3DBAD1EEB9D545F7B29C868C5FAEBBE9, vertexId=vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,361 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3DBAD1EEB9D545F7B29C868C5FAEBBE9, vertexName=EF38E69CA5A64637B85AD5AA61F789EF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:28:02,361 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3DBAD1EEB9D545F7B29C868C5FAEBBE9, vertexId=vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:02,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EF38E69CA5A64637B85AD5AA61F789EF, vertexId=vertex_1463524082182_0001_1_01, initRequestedTime=1463524082352, initedTime=1463524082363, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:28:02,363 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:28:02,363 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:02,363 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {CCAFF66EB8A84A40B2824E412BE2285B=forAllWorkUnits=true, update=[1]}
2016-05-17 22:28:02,364 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] parallelism set to 1
2016-05-17 22:28:02,364 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:02,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524082182_0001_1_00, eventCount=1
2016-05-17 22:28:02,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D6FB59917DDD419D85F83818C013C301, vertexId=vertex_1463524082182_0001_1_00, initRequestedTime=1463524082351, initedTime=1463524082365, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:28:02,365 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] with distanceFromRoot: 0
2016-05-17 22:28:02,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:02,365 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:28:02,366 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,366 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524082182_0001_1_00, startRequestedTime=1463524082366, startedTime=1463524082366
2016-05-17 22:28:02,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:02,366 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:02,367 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:02,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:TASK_STARTED]: vertexName=D6FB59917DDD419D85F83818C013C301, taskId=task_1463524082182_0001_1_00_000000, scheduledTime=1463524082367, launchTime=1463524082367
2016-05-17 22:28:02,367 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082182_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:02,368 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524082182_0001_1_00 for vertex: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF] numStartedSources: 1 numSources: 1
2016-05-17 22:28:02,368 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF] with distanceFromRoot: 1
2016-05-17 22:28:02,368 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,368 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D6FB59917DDD419D85F83818C013C301 in vertex: EF38E69CA5A64637B85AD5AA61F789EF
2016-05-17 22:28:02,369 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EF38E69CA5A64637B85AD5AA61F789EF with 1 source tasks and 1 pending tasks
2016-05-17 22:28:02,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524082182_0001_1_01, startRequestedTime=1463524082368, startedTime=1463524082368
2016-05-17 22:28:02,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:02,369 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:02,370 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C] test/(1/1) ...tGroupAssertionsPass/pass, VertexName: D6FB59917DDD419D85F83818C013C301, VertexParallelism: 1, TaskAttemptID:attempt_1463524082182_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CCAFF66EB8A84A40B2824E412BE2285B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EF38E69CA5A64637B85AD5AA61F789EF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:02,370 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082182_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:02,370 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:02,371 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524082182_0001, with dag id: dag_1463524082182_0001_1
2016-05-17 22:28:02,373 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:28:02,373 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:28:02,373 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:28:02,373 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524082182_0001_00_000001
2016-05-17 22:28:02,374 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:02,375 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524082182_0001_00_000001
2016-05-17 22:28:02,375 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:02,378 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:02,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:02,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524082182_0001_00_000001, launchTime=1463524082378
2016-05-17 22:28:02,379 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524082182_0001_00_000001
2016-05-17 22:28:02,380 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524082182_0001_1_00_000000_0] to container: [container_1463524082182_0001_00_000001]
2016-05-17 22:28:02,380 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:02,380 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524082182_0001_1_00_000000_0] started. Is using containerId: [container_1463524082182_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:28:02,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D6FB59917DDD419D85F83818C013C301, taskAttemptId=attempt_1463524082182_0001_1_00_000000_0, startTime=1463524082380, containerId=container_1463524082182_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524082182_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:28:02,380 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082182_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:02,381 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082182_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:02,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:02,381 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524082182_0001_00_000001 given task: attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02,381 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524082182_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02 Starting to run new task attempt: attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02,382 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02,382 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:02,382 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:02,383 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:02,383 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:02,383 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:02,383 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:02,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C] test/(1/1) ...tGroupAssertionsPass/pass, VertexName: D6FB59917DDD419D85F83818C013C301, VertexParallelism: 1, TaskAttemptID:attempt_1463524082182_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CCAFF66EB8A84A40B2824E412BE2285B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EF38E69CA5A64637B85AD5AA61F789EF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:02,384 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:02,384 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:02,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:02,385 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524082182_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:28:02,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CCAFF66EB8A84A40B2824E412BE2285B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:28:02,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:02,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:02,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CCAFF66EB8A84A40B2824E412BE2285B
2016-05-17 22:28:02,387 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: CCAFF66EB8A84A40B2824E412BE2285B
2016-05-17 22:28:02,389 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:02,390 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:28:02,390 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: CCAFF66EB8A84A40B2824E412BE2285B
2016-05-17 22:28:02,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CCAFF66EB8A84A40B2824E412BE2285B
2016-05-17 22:28:02,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EF38E69CA5A64637B85AD5AA61F789EF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:28:02,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:02,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EF38E69CA5A64637B85AD5AA61F789EF
2016-05-17 22:28:02,396 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D6FB59917DDD419D85F83818C013C301, ordinal: 0
2016-05-17 22:28:02,396 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D6FB59917DDD419D85F83818C013C301, mem on start (mb), free: 264, total: 479, max: 672
2016-05-17 22:28:02,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:02,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:02,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:02,397 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:28:02,397 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EF38E69CA5A64637B85AD5AA61F789EF
2016-05-17 22:28:02,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EF38E69CA5A64637B85AD5AA61F789EF
2016-05-17 22:28:02,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:02,398 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:02,398 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:28:02,398 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:02,398 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:28:02,399 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:02,399 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:28:02,399 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CCAFF66EB8A84A40B2824E412BE2285B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:28:02,399 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EF38E69CA5A64637B85AD5AA61F789EF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:28:02,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:02,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CCAFF66EB8A84A40B2824E412BE2285B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:02,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CCAFF66EB8A84A40B2824E412BE2285B
2016-05-17 22:28:02,400 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:02,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CCAFF66EB8A84A40B2824E412BE2285B
2016-05-17 22:28:02,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:02,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:02,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:02,401 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02,402 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:28:02,403 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@35333295
2016-05-17 22:28:02,405 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@35333295
2016-05-17 22:28:02,405 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:28:02,405 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:28:02,407 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:02,408 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: CCAFF66EB8A84A40B2824E412BE2285B
2016-05-17 22:28:02,408 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: 61D54142ADBE42BE93EF862208BB9B77
2016-05-17 22:28:02,409 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] 61D54142ADBE42BE93EF862208BB9B77
2016-05-17 22:28:02,410 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6fa0492; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@84210d2; comparator=cascading.tuple.hadoop.util.TupleComparator@295e4f3a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:28:02,413 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:28:02,413 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:28:02,413 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:02,601 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:28:02,601 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:28:02,602 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:28:02,602 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:28:02,602 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:28:02,604 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:02,604 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:02,604 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D6FB59917DDD419D85F83818C013C301, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:28:02,608 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02,608 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D6FB59917DDD419D85F83818C013C301, mem on close (mb), free: 456, total: 579, max: 672
2016-05-17 22:28:02,609 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:28:02,609 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:28:02,609 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:28:02,610 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:28:02,612 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:28:02,612 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:28:02,613 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:28:02 Completed running task attempt: attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02,616 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524082182_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:02,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524082182_0001_1_00, eventCount=1
2016-05-17 22:28:02,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=175, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=607125504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524082605, Process_Duration=3, Process_End_Time=1463524082608, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:28:02,617 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:02,618 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:02,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:02,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D6FB59917DDD419D85F83818C013C301, taskAttemptId=attempt_1463524082182_0001_1_00_000000_0, startTime=1463524082380, finishTime=1463524082617, timeTaken=237, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=175, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=607125504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524082605, Process_Duration=3, Process_End_Time=1463524082608, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:02,619 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082182_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:02,619 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524082182_0001_1_00_000000_0
2016-05-17 22:28:02,619 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:02,619 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:02,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:TASK_FINISHED]: vertexName=D6FB59917DDD419D85F83818C013C301, taskId=task_1463524082182_0001_1_00_000000, startTime=1463524082380, finishTime=1463524082619, timeTaken=239, status=SUCCEEDED, successfulAttemptID=attempt_1463524082182_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=175, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=607125504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524082605, Process_Duration=3, Process_End_Time=1463524082608, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:02,620 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082182_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:02,621 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] : 1
2016-05-17 22:28:02,621 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524082182_0001_00_000001
2016-05-17 22:28:02,621 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:02,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:VERTEX_FINISHED]: vertexName=D6FB59917DDD419D85F83818C013C301, vertexId=vertex_1463524082182_0001_1_00, initRequestedTime=1463524082351, initedTime=1463524082365, startRequestedTime=1463524082366, startedTime=1463524082366, finishTime=1463524082622, timeTaken=256, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=175, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=607125504, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524082605, Process_Duration=3, Process_End_Time=1463524082608, Read_Duration=0, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524082380, firstTasksToStart=[ task_1463524082182_0001_1_00_000000 ], lastTaskFinishTime=1463524082617, lastTasksToFinish=[ task_1463524082182_0001_1_00_000000 ], minTaskDuration=237, maxTaskDuration=237, avgTaskDuration=237.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524082182_0001_1_00_000000 ], longestDurationTasks=[ task_1463524082182_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:02,623 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:02,623 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:02,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524082182_0001_00_000001, stoppedTime=1463524082623, exitStatus=0
2016-05-17 22:28:02,623 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:02,623 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:02,624 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF] attempt: attempt_1463524082182_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:28:02,624 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EF38E69CA5A64637B85AD5AA61F789EF
2016-05-17 22:28:02,624 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,624 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:02,624 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524082182_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:28:02,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:TASK_STARTED]: vertexName=EF38E69CA5A64637B85AD5AA61F789EF, taskId=task_1463524082182_0001_1_01_000000, scheduledTime=1463524082624, launchTime=1463524082624
2016-05-17 22:28:02,625 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:02,625 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082182_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:02,625 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:28:02,625 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,625 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C] test/(1/1) ...tGroupAssertionsPass/pass, VertexName: EF38E69CA5A64637B85AD5AA61F789EF, VertexParallelism: 1, TaskAttemptID:attempt_1463524082182_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D6FB59917DDD419D85F83818C013C301, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3DBAD1EEB9D545F7B29C868C5FAEBBE9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:02,625 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082182_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:02,626 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:02,626 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524082182_0001_00_000002
2016-05-17 22:28:02,626 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:02,628 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524082182_0001_00_000002
2016-05-17 22:28:02,628 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:02,629 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:02,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:02,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524082182_0001_00_000002, launchTime=1463524082629
2016-05-17 22:28:02,630 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524082182_0001_00_000002
2016-05-17 22:28:02,630 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524082182_0001_1_01_000000_0] to container: [container_1463524082182_0001_00_000002]
2016-05-17 22:28:02,630 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:02,630 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524082182_0001_00_000002 given task: attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,630 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524082182_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,631 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524082182_0001_1_01_000000_0] started. Is using containerId: [container_1463524082182_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:28:02,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EF38E69CA5A64637B85AD5AA61F789EF, taskAttemptId=attempt_1463524082182_0001_1_01_000000_0, startTime=1463524082630, containerId=container_1463524082182_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524082182_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:28:02,631 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082182_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:02,631 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082182_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:02,631 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:02,632 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524082182_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:28:02 Starting to run new task attempt: attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,632 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,633 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:02,633 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:02,632 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524082182_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:28:02,634 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:02,634 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524082182_0001_00_000001, Exiting
2016-05-17 22:28:02,634 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524082182_0001_00_000001
2016-05-17 22:28:02,634 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524082182_0001_00_000001
2016-05-17 22:28:02,634 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:02,635 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:02,635 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:02,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C] test/(1/1) ...tGroupAssertionsPass/pass, VertexName: EF38E69CA5A64637B85AD5AA61F789EF, VertexParallelism: 1, TaskAttemptID:attempt_1463524082182_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D6FB59917DDD419D85F83818C013C301, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3DBAD1EEB9D545F7B29C868C5FAEBBE9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:02,636 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:02,637 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:02,637 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524082182_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:28:02,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:02,637 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524082182_0001_00_000001 completed successfully
2016-05-17 22:28:02,638 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:28:02,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:02,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D6FB59917DDD419D85F83818C013C301, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:28:02,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:02,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,640 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524082182_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:28:02,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3DBAD1EEB9D545F7B29C868C5FAEBBE9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:28:02,641 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:02,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:02,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3DBAD1EEB9D545F7B29C868C5FAEBBE9
2016-05-17 22:28:02,641 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:28:02,642 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:28:02,642 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:28:02,642 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,642 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3DBAD1EEB9D545F7B29C868C5FAEBBE9
2016-05-17 22:28:02,650 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EF38E69CA5A64637B85AD5AA61F789EF, ordinal: 1
2016-05-17 22:28:02,650 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EF38E69CA5A64637B85AD5AA61F789EF, mem on start (mb), free: 450, total: 579, max: 672
2016-05-17 22:28:02,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:02,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:02,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:02,656 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:28:02,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3DBAD1EEB9D545F7B29C868C5FAEBBE9
2016-05-17 22:28:02,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:02,656 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:02,656 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:28:02,657 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:28:02,657 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:02,657 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:02,657 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:28:02,657 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D6FB59917DDD419D85F83818C013C301, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 22:28:02,657 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3DBAD1EEB9D545F7B29C868C5FAEBBE9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:28:02,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:02,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,658 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:02,658 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:28:02,659 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D6FB59917DDD419D85F83818C013C301 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:28:02,659 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 22:28:02,659 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:28:02,659 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:28:02,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D6FB59917DDD419D85F83818C013C301 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:02,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:02,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:02,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:02,662 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,663 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524082182_0001_1_00_000000_0_10069, runDuration: 220889, ]
2016-05-17 22:28:02,665 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524082182_0001_1_00_000000_0_10069] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524082664, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 22:28:02,665 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,665 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:28:02,665 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:28:02,666 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D6FB59917DDD419D85F83818C013C301] #12 in 2ms
2016-05-17 22:28:02,666 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: 61D54142ADBE42BE93EF862208BB9B77
2016-05-17 22:28:02,666 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D6FB59917DDD419D85F83818C013C301] #12, status:false, isInterrupted:false
2016-05-17 22:28:02,667 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsPass/pass"], id: 3DBAD1EEB9D545F7B29C868C5FAEBBE9
2016-05-17 22:28:02,668 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsPass/pass"]
2016-05-17 22:28:02,668 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] 61D54142ADBE42BE93EF862208BB9B77, for 1 inputs
2016-05-17 22:28:02,670 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:28:02,670 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:28:02,670 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:02,670 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:28:02,671 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:28:02,671 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,671 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:28:02,671 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EF38E69CA5A64637B85AD5AA61F789EF, all 1 inputs ready in: 00:00:00.003
2016-05-17 22:28:02,673 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@516b0351; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@48a6e499
2016-05-17 22:28:02,675 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:28:02,675 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:28:02,675 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,675 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524082182_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:28:02,676 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,676 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EF38E69CA5A64637B85AD5AA61F789EF, mem on close (mb), free: 447, total: 579, max: 672
2016-05-17 22:28:02,677 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D6FB59917DDD419D85F83818C013C301
2016-05-17 22:28:02,677 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:28:02,677 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:28:02,677 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:02,678 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:28:02,678 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:28:02 Completed running task attempt: attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,678 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524082182_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:02,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=607125504, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524082672, Process_Duration=4, Process_End_Time=1463524082676, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=1]]
2016-05-17 22:28:02,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:02,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EF38E69CA5A64637B85AD5AA61F789EF, taskAttemptId=attempt_1463524082182_0001_1_01_000000_0, startTime=1463524082630, finishTime=1463524082679, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=607125504, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524082672, Process_Duration=4, Process_End_Time=1463524082676, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:28:02,681 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082182_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:02,680 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:02,681 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:02,681 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524082182_0001_1_01_000000_0
2016-05-17 22:28:02,681 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524082182_0001_00_000002
2016-05-17 22:28:02,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:02,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:TASK_FINISHED]: vertexName=EF38E69CA5A64637B85AD5AA61F789EF, taskId=task_1463524082182_0001_1_01_000000, startTime=1463524082630, finishTime=1463524082681, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463524082182_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=607125504, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524082672, Process_Duration=4, Process_End_Time=1463524082676, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:28:02,682 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082182_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:02,682 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF] : 1
2016-05-17 22:28:02,682 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:02,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:VERTEX_FINISHED]: vertexName=EF38E69CA5A64637B85AD5AA61F789EF, vertexId=vertex_1463524082182_0001_1_01, initRequestedTime=1463524082352, initedTime=1463524082363, startRequestedTime=1463524082368, startedTime=1463524082368, finishTime=1463524082682, timeTaken=314, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=607125504, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524082672, Process_Duration=4, Process_End_Time=1463524082676, Read_Duration=1, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1, vertexStats=firstTaskStartTime=1463524082630, firstTasksToStart=[ task_1463524082182_0001_1_01_000000 ], lastTaskFinishTime=1463524082679, lastTasksToFinish=[ task_1463524082182_0001_1_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524082182_0001_1_01_000000 ], longestDurationTasks=[ task_1463524082182_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:02,683 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:02,684 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:02,684 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524082182_0001_1
2016-05-17 22:28:02,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524082182_0001_1
2016-05-17 22:28:02,684 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3DBAD1EEB9D545F7B29C868C5FAEBBE9 for vertex: vertex_1463524082182_0001_1_01 [EF38E69CA5A64637B85AD5AA61F789EF]
2016-05-17 22:28:02,681 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:02,691 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463524082182_0001_1_00 [D6FB59917DDD419D85F83818C013C301]
2016-05-17 22:28:02,693 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524082182_0001_1, queueSize=0
2016-05-17 22:28:02,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:28:02,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524082182_0001_1, startTime=1463524082350, finishTime=1463524082691, timeTaken=341, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2891, FILE_BYTES_WRITTEN=704, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=14, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=175, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1214251008, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=8, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927048165277, Process_Duration=7, Process_End_Time=2927048165284, Read_Duration=1, Tuples_Read=17, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=1
2016-05-17 22:28:02,694 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524082182_0001_1 finished with state: SUCCEEDED
2016-05-17 22:28:02,694 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524082182_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:28:02,696 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:02,696 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:02,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082182_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524082182_0001_00_000002, stoppedTime=1463524082696, exitStatus=0
2016-05-17 22:28:02,697 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:02 Completed Dag: dag_1463524082182_0001_1
2016-05-17 22:28:02,698 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524082182_0001_1, dagState=SUCCEEDED
2016-05-17 22:28:02,698 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524082182_0001_1_post
2016-05-17 22:28:02,698 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:02,699 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:28:02,699 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524082182_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 22:28:02,700 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082182_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:28:02,702 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tGroupAssertionsPass/pass, applicationId=application_1463524082182_0001
2016-05-17 22:28:02,706 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:28:02,706 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:28:02,706 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:28:02,706 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:28:02,706 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:28:02,708 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:28:02,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:28:02,709 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:28:02,709 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:28:02,709 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:28:02,709 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:28:02,709 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:28:02,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:28:02,711 INFO task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1463524082182_0001_00_000002
2016-05-17 22:28:02,711 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524082182_0001_00_000002
2016-05-17 22:28:02,711 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524082182_0001_00_000002
2016-05-17 22:28:02,719 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/3D14FC727A524A0891D41803B5BD1538/A71FF201C5BD43009FB60E9B87A41F2C/.tez/application_1463524082182_0001
2016-05-17 22:28:02,719 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testGroupAssertionsPass/pass/_temporary
2016-05-17 22:28:02,720 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.557
2016-05-17 22:28:02,756 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testGroupAssertionsPass/pass
2016-05-17 22:28:02,779 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:28:02,779 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:28:02,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 22:28:02,839 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.036
2016-05-17 22:28:02,839 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:02,840 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:02,840 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:28:02,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:28:02,869 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsFail/value/fail
2016-05-17 22:28:02,939 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:28:02,939 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:02,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsFail/value/fail"]
2016-05-17 22:28:02,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:28:02,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:28:02,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:28:02,941 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...AssertionsFail/value/fail
2016-05-17 22:28:02,941 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:28:02,953 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:28:02,958 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:02,958 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:28:02,958 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:28:02,970 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2/.tez/application_1463524082958_0001 doesn't exist and is created
2016-05-17 22:28:02,992 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:28:02,993 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:28:02,993 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2/.tez/application_1463524082958_0001
2016-05-17 22:28:02,999 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524082958_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:03,015 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:28:03,016 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:28:03,018 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:28:03,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:28:03,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2/.tez/application_1463524082958_0001/localmode-log-dir
2016-05-17 22:28:03,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2/.tez/application_1463524082958_0001/localmode-log-dir/history.txt.appattempt_1463524082958_0001_000000, maxErrors=10
2016-05-17 22:28:03,020 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:28:03,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524082958_0001_000000, appSubmitTime=1463524082998, launchTime=1463524082998
2016-05-17 22:28:03,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:28:03,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:28:03,031 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59354
2016-05-17 22:28:03,033 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:28:03,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:28:03,034 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:28:03,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524082958_0001_000000, startTime=1463524083036
2016-05-17 22:28:03,037 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:28:03,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:28:03,093 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:28:03,093 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:28:03,093 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...AssertionsFail/value/fail, applicationId=application_1463524082958_0001, dagName=[6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2] test/(1/1) ...AssertionsFail/value/fail
2016-05-17 22:28:03,111 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2] test/(1/1) ...AssertionsFail/value/fail
2016-05-17 22:28:03,113 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524082958_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2/.tez/application_1463524082958_0001/localmode-log-dir/dag_1463524082958_0001_1.dot
2016-05-17 22:28:03,113 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2/.tez/application_1463524082958_0001/localmode-log-dir/dag_1463524082958_0001_1-tez-dag.pb.txt
2016-05-17 22:28:03,120 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524082958_0001_1
2016-05-17 22:28:03,120 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:03,120 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2] test/(1/1) ...AssertionsFail/value/fail
2016-05-17 22:28:03 Running Dag: dag_1463524082958_0001_1
2016-05-17 22:28:03,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524082958_0001_1, submitTime=1463524083111
2016-05-17 22:28:03,136 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:28:03,137 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F7A8D10C483D4EB1B5608508659A1CAC
2016-05-17 22:28:03,137 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:28:03,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:28:03,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524082958_0001_1, initTime=1463524083136
2016-05-17 22:28:03,138 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524082958_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:28:03,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:28:03,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:DAG_STARTED]: dagID=dag_1463524082958_0001_1, startTime=1463524083138
2016-05-17 22:28:03,138 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:28:03,139 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524082958_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:28:03,139 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...AssertionsFail/value/fail, applicationId=application_1463524082958_0001, dagName=[6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2] test/(1/1) ...AssertionsFail/value/fail
2016-05-17 22:28:03,139 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C24377DF97A44BAEA9D8E50F90C4AADA : {0B9F2FDC04194B0FB7A5CA235C6A02FB={InputName=0B9F2FDC04194B0FB7A5CA235C6A02FB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:28:03,139 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0B9F2FDC04194B0FB7A5CA235C6A02FB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:28:03,139 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,139 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,139 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,140 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]. Starting root input initializers: 1
2016-05-17 22:28:03,140 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:28:03,140 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,143 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:28:03,143 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,143 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,144 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,144 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=152FD1B06B824BCCBEFD19571EACCFD3, vertexId=vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:28:03,144 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=152FD1B06B824BCCBEFD19571EACCFD3, vertexId=vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,146 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=152FD1B06B824BCCBEFD19571EACCFD3, vertexName=F7A8D10C483D4EB1B5608508659A1CAC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:28:03,146 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=152FD1B06B824BCCBEFD19571EACCFD3, vertexId=vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:03,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F7A8D10C483D4EB1B5608508659A1CAC, vertexId=vertex_1463524082958_0001_1_01, initRequestedTime=1463524083140, initedTime=1463524083147, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:28:03,147 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:28:03,148 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0B9F2FDC04194B0FB7A5CA235C6A02FB on vertex vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,149 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:28:03,149 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0B9F2FDC04194B0FB7A5CA235C6A02FB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 22:28:03,150 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:28:03,150 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:28:03,151 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:28:03,152 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0B9F2FDC04194B0FB7A5CA235C6A02FB on vertex vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,152 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:03,152 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0B9F2FDC04194B0FB7A5CA235C6A02FB=forAllWorkUnits=true, update=[1]}
2016-05-17 22:28:03,153 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] parallelism set to 1
2016-05-17 22:28:03,153 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:03,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524082958_0001_1_00, eventCount=1
2016-05-17 22:28:03,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C24377DF97A44BAEA9D8E50F90C4AADA, vertexId=vertex_1463524082958_0001_1_00, initRequestedTime=1463524083139, initedTime=1463524083153, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:28:03,155 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] with distanceFromRoot: 0
2016-05-17 22:28:03,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:03,155 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:28:03,156 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,156 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524082958_0001_1_00, startRequestedTime=1463524083156, startedTime=1463524083156
2016-05-17 22:28:03,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:03,157 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:03,158 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:03,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:TASK_STARTED]: vertexName=C24377DF97A44BAEA9D8E50F90C4AADA, taskId=task_1463524082958_0001_1_00_000000, scheduledTime=1463524083158, launchTime=1463524083158
2016-05-17 22:28:03,159 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082958_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:03,159 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524082958_0001_1_00 for vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] numStartedSources: 1 numSources: 1
2016-05-17 22:28:03,160 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] with distanceFromRoot: 1
2016-05-17 22:28:03,160 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,161 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C24377DF97A44BAEA9D8E50F90C4AADA in vertex: F7A8D10C483D4EB1B5608508659A1CAC
2016-05-17 22:28:03,161 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F7A8D10C483D4EB1B5608508659A1CAC with 1 source tasks and 1 pending tasks
2016-05-17 22:28:03,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:03,162 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524082958_0001_1_01, startRequestedTime=1463524083160, startedTime=1463524083160
2016-05-17 22:28:03,162 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:03,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524082958_0001, with dag id: dag_1463524082958_0001_1
2016-05-17 22:28:03,162 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2] test/(1/1) ...AssertionsFail/value/fail, VertexName: C24377DF97A44BAEA9D8E50F90C4AADA, VertexParallelism: 1, TaskAttemptID:attempt_1463524082958_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0B9F2FDC04194B0FB7A5CA235C6A02FB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F7A8D10C483D4EB1B5608508659A1CAC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:03,163 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082958_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:03,163 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:03,164 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:28:03,164 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:28:03,164 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:28:03,165 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524082958_0001_00_000001
2016-05-17 22:28:03,166 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:03,167 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524082958_0001_00_000001
2016-05-17 22:28:03,167 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:03,168 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:03,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524082958_0001_00_000001, launchTime=1463524083168
2016-05-17 22:28:03,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:03,169 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524082958_0001_00_000001
2016-05-17 22:28:03,170 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524082958_0001_1_00_000000_0] to container: [container_1463524082958_0001_00_000001]
2016-05-17 22:28:03,170 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:03,170 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524082958_0001_1_00_000000_0] started. Is using containerId: [container_1463524082958_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:28:03,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C24377DF97A44BAEA9D8E50F90C4AADA, taskAttemptId=attempt_1463524082958_0001_1_00_000000_0, startTime=1463524083170, containerId=container_1463524082958_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524082958_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:28:03,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:03,170 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524082958_0001_00_000001 given task: attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03,170 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082958_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:03,171 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082958_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:03,171 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524082958_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03 Starting to run new task attempt: attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03,171 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03,171 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:03,171 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:03,171 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:03,172 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:03,172 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:03,172 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:03,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2] test/(1/1) ...AssertionsFail/value/fail, VertexName: C24377DF97A44BAEA9D8E50F90C4AADA, VertexParallelism: 1, TaskAttemptID:attempt_1463524082958_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0B9F2FDC04194B0FB7A5CA235C6A02FB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F7A8D10C483D4EB1B5608508659A1CAC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:03,172 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 22:28:03,173 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:03,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,175 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524082958_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:28:03,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0B9F2FDC04194B0FB7A5CA235C6A02FB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:28:03,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F7A8D10C483D4EB1B5608508659A1CAC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:28:03,175 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:03,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0B9F2FDC04194B0FB7A5CA235C6A02FB
2016-05-17 22:28:03,176 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0B9F2FDC04194B0FB7A5CA235C6A02FB
2016-05-17 22:28:03,179 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:28:03,179 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:03,179 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0B9F2FDC04194B0FB7A5CA235C6A02FB
2016-05-17 22:28:03,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0B9F2FDC04194B0FB7A5CA235C6A02FB
2016-05-17 22:28:03,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:03,180 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F7A8D10C483D4EB1B5608508659A1CAC
2016-05-17 22:28:03,182 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:28:03,182 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F7A8D10C483D4EB1B5608508659A1CAC
2016-05-17 22:28:03,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F7A8D10C483D4EB1B5608508659A1CAC
2016-05-17 22:28:03,191 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C24377DF97A44BAEA9D8E50F90C4AADA, ordinal: 0
2016-05-17 22:28:03,191 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C24377DF97A44BAEA9D8E50F90C4AADA, mem on start (mb), free: 363, total: 579, max: 672
2016-05-17 22:28:03,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:03,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:03,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:03,192 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:03,192 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:28:03,193 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:03,193 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:28:03,193 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:03,193 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:28:03,194 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0B9F2FDC04194B0FB7A5CA235C6A02FB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:28:03,194 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F7A8D10C483D4EB1B5608508659A1CAC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:28:03,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:03,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0B9F2FDC04194B0FB7A5CA235C6A02FB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:03,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:03,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:03,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0B9F2FDC04194B0FB7A5CA235C6A02FB
2016-05-17 22:28:03,195 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:03,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0B9F2FDC04194B0FB7A5CA235C6A02FB
2016-05-17 22:28:03,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:03,196 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03,196 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:28:03,196 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@752701ec
2016-05-17 22:28:03,197 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@752701ec
2016-05-17 22:28:03,197 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:28:03,197 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:28:03,200 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:03,201 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 0B9F2FDC04194B0FB7A5CA235C6A02FB
2016-05-17 22:28:03,201 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: 9E64657DA930442AB0492B1E752659CA
2016-05-17 22:28:03,201 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] 9E64657DA930442AB0492B1E752659CA
2016-05-17 22:28:03,202 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3d289d5e; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3573a5b0; comparator=cascading.tuple.hadoop.util.TupleComparator@3b82e13a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:28:03,202 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:28:03,203 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:28:03,203 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:03,331 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:28:03,332 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:28:03,344 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:28:03,344 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:28:03,344 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:28:03,344 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:28:03,344 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:28:03,345 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:03,345 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:03,345 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C24377DF97A44BAEA9D8E50F90C4AADA, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:28:03,348 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03,348 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C24377DF97A44BAEA9D8E50F90C4AADA, mem on close (mb), free: 559, total: 685, max: 685
2016-05-17 22:28:03,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:28:03,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:28:03,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:28:03,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:28:03,352 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:28:03 Completed running task attempt: attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03,355 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524082958_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:03,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524082958_0001_1_00, eventCount=1
2016-05-17 22:28:03,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=128, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=718798848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524083346, Process_Duration=2, Process_End_Time=1463524083348, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:28:03,356 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:03,356 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:03,356 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524082958_0001_00_000001
2016-05-17 22:28:03,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C24377DF97A44BAEA9D8E50F90C4AADA, taskAttemptId=attempt_1463524082958_0001_1_00_000000_0, startTime=1463524083170, finishTime=1463524083356, timeTaken=186, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=128, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=718798848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524083346, Process_Duration=2, Process_End_Time=1463524083348, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:03,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:03,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082958_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:03,357 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524082958_0001_1_00_000000_0
2016-05-17 22:28:03,357 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:03,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:TASK_FINISHED]: vertexName=C24377DF97A44BAEA9D8E50F90C4AADA, taskId=task_1463524082958_0001_1_00_000000, startTime=1463524083170, finishTime=1463524083357, timeTaken=187, status=SUCCEEDED, successfulAttemptID=attempt_1463524082958_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=128, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=718798848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524083346, Process_Duration=2, Process_End_Time=1463524083348, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:03,358 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082958_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:03,358 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] : 1
2016-05-17 22:28:03,358 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:03,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:VERTEX_FINISHED]: vertexName=C24377DF97A44BAEA9D8E50F90C4AADA, vertexId=vertex_1463524082958_0001_1_00, initRequestedTime=1463524083139, initedTime=1463524083153, startRequestedTime=1463524083156, startedTime=1463524083156, finishTime=1463524083358, timeTaken=202, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=128, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=718798848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524083346, Process_Duration=2, Process_End_Time=1463524083348, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524083170, firstTasksToStart=[ task_1463524082958_0001_1_00_000000 ], lastTaskFinishTime=1463524083356, lastTasksToFinish=[ task_1463524082958_0001_1_00_000000 ], minTaskDuration=186, maxTaskDuration=186, avgTaskDuration=186.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524082958_0001_1_00_000000 ], longestDurationTasks=[ task_1463524082958_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:03,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:03,359 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:03,360 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:03,360 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] attempt: attempt_1463524082958_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:28:03,360 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F7A8D10C483D4EB1B5608508659A1CAC
2016-05-17 22:28:03,360 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524082958_0001_00_000001, stoppedTime=1463524083360, exitStatus=0
2016-05-17 22:28:03,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:03,360 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:03,361 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:03,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:TASK_STARTED]: vertexName=F7A8D10C483D4EB1B5608508659A1CAC, taskId=task_1463524082958_0001_1_01_000000, scheduledTime=1463524083361, launchTime=1463524083361
2016-05-17 22:28:03,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:03,362 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082958_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:03,362 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,362 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524082958_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:28:03,362 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2] test/(1/1) ...AssertionsFail/value/fail, VertexName: F7A8D10C483D4EB1B5608508659A1CAC, VertexParallelism: 1, TaskAttemptID:attempt_1463524082958_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C24377DF97A44BAEA9D8E50F90C4AADA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=152FD1B06B824BCCBEFD19571EACCFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:03,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082958_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:03,363 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:28:03,363 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:03,363 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524082958_0001_00_000002
2016-05-17 22:28:03,364 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:03,365 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524082958_0001_00_000002
2016-05-17 22:28:03,365 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:03,366 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:03,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524082958_0001_00_000002, launchTime=1463524083366
2016-05-17 22:28:03,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:03,366 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524082958_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:28:03,367 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524082958_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:28:03,367 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524082958_0001_00_000002
2016-05-17 22:28:03,367 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524082958_0001_00_000001, Exiting
2016-05-17 22:28:03,367 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524082958_0001_00_000001
2016-05-17 22:28:03,367 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524082958_0001_00_000001
2016-05-17 22:28:03,368 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524082958_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:28:03,368 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524082958_0001_00_000001 completed successfully
2016-05-17 22:28:03,368 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524082958_0001_1_01_000000_0] to container: [container_1463524082958_0001_00_000002]
2016-05-17 22:28:03,369 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:03,369 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524082958_0001_00_000002 given task: attempt_1463524082958_0001_1_01_000000_0
2016-05-17 22:28:03,369 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524082958_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524082958_0001_1_01_000000_0
2016-05-17 22:28:03,369 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:28:03 Starting to run new task attempt: attempt_1463524082958_0001_1_01_000000_0
2016-05-17 22:28:03,370 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524082958_0001_1_01_000000_0
2016-05-17 22:28:03,370 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:03,370 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:03,370 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:03,370 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:03,370 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:03,370 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:03,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2] test/(1/1) ...AssertionsFail/value/fail, VertexName: F7A8D10C483D4EB1B5608508659A1CAC, VertexParallelism: 1, TaskAttemptID:attempt_1463524082958_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C24377DF97A44BAEA9D8E50F90C4AADA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=152FD1B06B824BCCBEFD19571EACCFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:03,371 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=718798848
2016-05-17 22:28:03,370 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524082958_0001_1_01_000000_0] started. Is using containerId: [container_1463524082958_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:28:03,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:03,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F7A8D10C483D4EB1B5608508659A1CAC, taskAttemptId=attempt_1463524082958_0001_1_01_000000_0, startTime=1463524083369, containerId=container_1463524082958_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524082958_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:28:03,371 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524082958_0001_1_01_000000_0
2016-05-17 22:28:03,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:03,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C24377DF97A44BAEA9D8E50F90C4AADA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:28:03,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:03,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=152FD1B06B824BCCBEFD19571EACCFD3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:28:03,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:03,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 152FD1B06B824BCCBEFD19571EACCFD3
2016-05-17 22:28:03,374 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:28:03,375 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 152FD1B06B824BCCBEFD19571EACCFD3
2016-05-17 22:28:03,374 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 646918976, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:28:03,375 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:28:03,376 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 646918976, type: INPUT, componentVertexName: C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,376 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:03,377 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524082958_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:28:03,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,377 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082958_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:03,379 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082958_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:03,384 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F7A8D10C483D4EB1B5608508659A1CAC, ordinal: 1
2016-05-17 22:28:03,384 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7A8D10C483D4EB1B5608508659A1CAC, mem on start (mb), free: 553, total: 685, max: 685
2016-05-17 22:28:03,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:03,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:03,390 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:28:03,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 152FD1B06B824BCCBEFD19571EACCFD3
2016-05-17 22:28:03,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:03,390 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:03,390 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:28:03,391 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:03,391 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:28:03,391 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:03,391 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 646918976, TotalRequestedScaled: 5.971559778461539E8, TotalJVMHeap: 718798848, TotalAvailable: 481595228, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:28:03,391 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 152FD1B06B824BCCBEFD19571EACCFD3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:28:03,392 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C24377DF97A44BAEA9D8E50F90C4AADA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=646918976, allocated=481595228
2016-05-17 22:28:03,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:03,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,393 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:03,393 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:28:03,393 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C24377DF97A44BAEA9D8E50F90C4AADA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:28:03,393 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=646918976, postMergeMem=0, RuntimeTotalAvailable=481595228. Updated to: ShuffleMem=481595228, postMergeMem: 0
2016-05-17 22:28:03,393 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=481595228, maxSingleShuffleLimit=120398808, mergeThreshold=433435712, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:28:03,394 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:28:03,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C24377DF97A44BAEA9D8E50F90C4AADA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:03,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:03,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:03,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:03,398 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524082958_0001_1_01_000000_0
2016-05-17 22:28:03,398 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524082958_0001_1_00_000000_0_10075, runDuration: 173597, ]
2016-05-17 22:28:03,400 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:28:03,401 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: 9E64657DA930442AB0492B1E752659CA
2016-05-17 22:28:03,401 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsFail/value/fail"], id: 152FD1B06B824BCCBEFD19571EACCFD3
2016-05-17 22:28:03,402 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsFail/value/fail"]
2016-05-17 22:28:03,402 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] 9E64657DA930442AB0492B1E752659CA, for 1 inputs
2016-05-17 22:28:03,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524082958_0001_1_00_000000_0_10075] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524083403, TimeTaken=2, Rate=0.31 MB/s
2016-05-17 22:28:03,404 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,404 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:28:03,404 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C24377DF97A44BAEA9D8E50F90C4AADA] #13 in 3ms
2016-05-17 22:28:03,405 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C24377DF97A44BAEA9D8E50F90C4AADA] #13, status:false, isInterrupted:false
2016-05-17 22:28:03,406 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:28:03,406 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:28:03,406 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:03,407 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:28:03,407 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:28:03,407 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,407 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:28:03,407 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F7A8D10C483D4EB1B5608508659A1CAC, all 1 inputs ready in: 00:00:00.005
2016-05-17 22:28:03,408 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@18182506; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5926b945
2016-05-17 22:28:03,410 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testValueAssertionsFail(AppliedAssertionsPlatformTest.java:111)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:03,410 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testValueAssertionsFail(AppliedAssertionsPlatformTest.java:111)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:03,412 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testValueAssertionsFail(AppliedAssertionsPlatformTest.java:111)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:03,414 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:28:03,414 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:28:03,414 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524082958_0001_1_01_000000_0
2016-05-17 22:28:03,415 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524082958_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:28:03,415 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463524082958_0001_1_01_000000_0
cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testValueAssertionsFail(AppliedAssertionsPlatformTest.java:111)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-05-17 22:28:03,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=718798848, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524083407, Process_Duration=8, Process_End_Time=1463524083415, Read_Duration=1, Tuples_Read=7]]
2016-05-17 22:28:03,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:03,417 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7A8D10C483D4EB1B5608508659A1CAC, mem on close (mb), free: 550, total: 685, max: 685
2016-05-17 22:28:03,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F7A8D10C483D4EB1B5608508659A1CAC, index=1
2016-05-17 22:28:03,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F7A8D10C483D4EB1B5608508659A1CAC, taskAttemptId=attempt_1463524082958_0001_1_01_000000_0, startTime=1463524083369, finishTime=1463524083417, timeTaken=48, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testValueAssertionsFail(AppliedAssertionsPlatformTest.java:111)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=718798848, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524083407, Process_Duration=8, Process_End_Time=1463524083415, Read_Duration=1, Tuples_Read=7
2016-05-17 22:28:03,418 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082958_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 22:28:03,418 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463524082958_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 22:28:03,418 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:03,418 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:TASK_FINISHED]: vertexName=F7A8D10C483D4EB1B5608508659A1CAC, taskId=task_1463524082958_0001_1_01_000000, startTime=1463524083369, finishTime=1463524083418, timeTaken=49, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testValueAssertionsFail(AppliedAssertionsPlatformTest.java:111)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-05-17 22:28:03,420 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524082958_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 22:28:03,420 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:03,420 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] : 1
2016-05-17 22:28:03,420 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] because task failed: task_1463524082958_0001_1_01_000000
2016-05-17 22:28:03,420 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:28:03,421 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:28:03,420 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] due to trigger: OWN_TASK_FAILURE
2016-05-17 22:28:03,421 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 22:28:03,421 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,421 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=152FD1B06B824BCCBEFD19571EACCFD3, vertexId=vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,421 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:03,422 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:28:03,422 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:28:03,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F7A8D10C483D4EB1B5608508659A1CAC, sourceVertex=C24377DF97A44BAEA9D8E50F90C4AADA
2016-05-17 22:28:03,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F7A8D10C483D4EB1B5608508659A1CAC, sourceVertex=152FD1B06B824BCCBEFD19571EACCFD3
2016-05-17 22:28:03,422 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524082958_0001_00_000002
2016-05-17 22:28:03,422 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524082958_0001_00_000002
2016-05-17 22:28:03,423 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524082958_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 22:28:03,423 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463524082958_0001_00_000002 completed but with errors
2016-05-17 22:28:03,423 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:03,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:VERTEX_FINISHED]: vertexName=F7A8D10C483D4EB1B5608508659A1CAC, vertexId=vertex_1463524082958_0001_1_01, initRequestedTime=1463524083140, initedTime=1463524083147, startRequestedTime=1463524083160, startedTime=1463524083160, finishTime=1463524083422, timeTaken=262, status=FAILED, diagnostics=Task failed, taskId=task_1463524082958_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testValueAssertionsFail(AppliedAssertionsPlatformTest.java:111)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463524083369, firstTasksToStart=[ task_1463524082958_0001_1_01_000000 ], lastTaskFinishTime=1463524083417, lastTasksToFinish=[ task_1463524082958_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 22:28:03,423 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 22:28:03,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524082958_0001_00_000002, stoppedTime=1463524083423, exitStatus=0
2016-05-17 22:28:03,423 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:03,423 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:03,424 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463524082958_0001_1_01_000000_0 failed: true container: container_1463524082958_0001_00_000002 numFailedTAs: 0
2016-05-17 22:28:03,424 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 22:28:03,424 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 22:28:03,424 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524082958_0001_1
2016-05-17 22:28:03,424 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463524082958_0001_1_00 [C24377DF97A44BAEA9D8E50F90C4AADA]
2016-05-17 22:28:03,424 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 152FD1B06B824BCCBEFD19571EACCFD3 for vertex: vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC]
2016-05-17 22:28:03,426 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524082958_0001_1, queueSize=0
2016-05-17 22:28:03,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:28:03,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524082958_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524082958_0001_1, startTime=1463524083138, finishTime=1463524083425, timeTaken=287, status=FAILED, diagnostics=Vertex failed, vertexName=F7A8D10C483D4EB1B5608508659A1CAC, vertexId=vertex_1463524082958_0001_1_01, diagnostics=[Task failed, taskId=task_1463524082958_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.operation.AssertionException: [cascading.operation.assertion.AppliedAssertionsPlatformTest.testValueAssertionsFail(AppliedAssertionsPlatformTest.java:111)] argument tuple: ['7'] was not equal to values: ['0']
at cascading.operation.assertion.BaseAssertion.throwFail(BaseAssertion.java:107)
at cascading.operation.assertion.BaseAssertion.fail(BaseAssertion.java:86)
at cascading.operation.assertion.AssertEquals.doAssert(AssertEquals.java:77)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:71)
at cascading.flow.stream.element.ValueAssertionEachStage.receive(ValueAssertionEachStage.java:35)
at cascading.flow.stream.duct.CloseReducingDuct.completeGroup(CloseReducingDuct.java:47)
at cascading.flow.stream.element.AggregatorEveryStage$1.collect(AggregatorEveryStage.java:69)
at cascading.tuple.TupleEntryCollector.safeCollect(TupleEntryCollector.java:145)
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:133)
at cascading.operation.aggregator.Count.complete(Count.java:82)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463524082958_0001_1_01 [F7A8D10C483D4EB1B5608508659A1CAC] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=128, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=718798848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524083346, Process_Duration=2, Process_End_Time=1463524083348, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:03,427 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524082958_0001_1 finished with state: FAILED
2016-05-17 22:28:03,427 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524082958_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:28:03,427 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463524082958_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 22:28:03,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524082958_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:28:03,427 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524082958_0001_00_000002
2016-05-17 22:28:03 Completed Dag: dag_1463524082958_0001_1
2016-05-17 22:28:03,428 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524082958_0001_1, dagState=FAILED
2016-05-17 22:28:03,428 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524082958_0001_1_post
2016-05-17 22:28:03,428 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:03,428 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:28:03,428 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524082958_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 22:28:03,441 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...AssertionsFail/value/fail, applicationId=application_1463524082958_0001
2016-05-17 22:28:03,443 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:28:03,444 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:28:03,444 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:28:03,444 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:28:03,444 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:28:03,444 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:28:03,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:28:03,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:28:03,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:28:03,445 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:28:03,445 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:28:03,446 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:28:03,446 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:28:03,450 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/6504F2ABF3984EBE9FCE3683BE204F28/FE0EF5CA1DBD407FA5A8B6C54B2D57A2/.tez/application_1463524082958_0001
2016-05-17 22:28:03,450 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping all jobs
2016-05-17 22:28:03,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopping: (1/1) ...AssertionsFail/value/fail
2016-05-17 22:28:03,451 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] stopped all jobs
2016-05-17 22:28:03,451 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsFail/value/fail/_temporary
2016-05-17 22:28:03,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.512
2016-05-17 22:28:03,452 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testValueAssertionsFail/value/fail
2016-05-17 22:28:03,479 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 22:28:03,479 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 22:28:03,519 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 22:28:03,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.033
2016-05-17 22:28:03,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:03,537 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 22:28:03,538 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 22:28:03,559 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-05-17 22:28:03,569 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsPass/value/pass
2016-05-17 22:28:03,636 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting
2016-05-17 22:28:03,637 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:03,637 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsPass/value/pass"]
2016-05-17 22:28:03,638 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] parallel execution of steps is enabled: false
2016-05-17 22:28:03,638 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] executing total steps: 1
2016-05-17 22:28:03,638 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] allocating management threads: 1
2016-05-17 22:28:03,640 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] starting step: (1/1) ...AssertionsPass/value/pass
2016-05-17 22:28:03,640 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 22:28:03,653 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] tez session mode enabled: true
2016-05-17 22:28:03,659 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:03,659 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 22:28:03,659 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 22:28:03,671 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20/.tez/application_1463524083659_0001 doesn't exist and is created
2016-05-17 22:28:03,687 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 22:28:03,687 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 22:28:03,688 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20/.tez/application_1463524083659_0001
2016-05-17 22:28:03,691 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463524083659_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 22:28:03,700 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 22:28:03,701 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 22:28:03,702 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 22:28:03,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 22:28:03,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20/.tez/application_1463524083659_0001/localmode-log-dir
2016-05-17 22:28:03,703 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20/.tez/application_1463524083659_0001/localmode-log-dir/history.txt.appattempt_1463524083659_0001_000000, maxErrors=10
2016-05-17 22:28:03,703 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 22:28:03,703 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463524083659_0001_000000, appSubmitTime=1463524083690, launchTime=1463524083690
2016-05-17 22:28:03,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 22:28:03,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 22:28:03,712 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 22:28:03,712 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 22:28:03,713 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:39316
2016-05-17 22:28:03,716 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 22:28:03,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463524083659_0001_000000, startTime=1463524083717
2016-05-17 22:28:03,717 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 22:28:03,717 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 22:28:03,788 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 22:28:03,788 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 22:28:03,788 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...AssertionsPass/value/pass, applicationId=application_1463524083659_0001, dagName=[56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20] test/(1/1) ...AssertionsPass/value/pass
2016-05-17 22:28:03,805 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20] test/(1/1) ...AssertionsPass/value/pass
2016-05-17 22:28:03,806 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463524083659_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20/.tez/application_1463524083659_0001/localmode-log-dir/dag_1463524083659_0001_1.dot
2016-05-17 22:28:03,807 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20/.tez/application_1463524083659_0001/localmode-log-dir/dag_1463524083659_0001_1-tez-dag.pb.txt
2016-05-17 22:28:03,815 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524083659_0001_1
2016-05-17 22:28:03,815 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:03,815 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20] test/(1/1) ...AssertionsPass/value/pass
2016-05-17 22:28:03 Running Dag: dag_1463524083659_0001_1
2016-05-17 22:28:03,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463524083659_0001_1, submitTime=1463524083805
2016-05-17 22:28:03,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 22:28:03,828 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F46378B714124D539310573022FADA58
2016-05-17 22:28:03,828 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 22:28:03,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463524083659_0001_1, initTime=1463524083827
2016-05-17 22:28:03,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 22:28:03,830 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524083659_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 22:28:03,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:DAG_STARTED]: dagID=dag_1463524083659_0001_1, startTime=1463524083830
2016-05-17 22:28:03,831 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 22:28:03,831 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524083659_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 22:28:03,831 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ECBD9F5C6AB4452CAC3DE7F303DB08FF : {B8DE3565C5434AD2A9D556FB4027A0CB={InputName=B8DE3565C5434AD2A9D556FB4027A0CB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 22:28:03,831 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B8DE3565C5434AD2A9D556FB4027A0CB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 22:28:03,831 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:03,831 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:03,831 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:03,832 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]. Starting root input initializers: 1
2016-05-17 22:28:03,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 22:28:03,832 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 22:28:03,832 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B8DE3565C5434AD2A9D556FB4027A0CB on vertex vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:03,834 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...AssertionsPass/value/pass, applicationId=application_1463524083659_0001, dagName=[56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20] test/(1/1) ...AssertionsPass/value/pass
2016-05-17 22:28:03,832 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:03,835 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 22:28:03,835 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B8DE3565C5434AD2A9D556FB4027A0CB asking for 1 tasks. Headroom: 685 Task Resource: 1024 waves: 1.7
2016-05-17 22:28:03,835 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 22:28:03,836 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 22:28:03,837 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-05-17 22:28:03,837 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B8DE3565C5434AD2A9D556FB4027A0CB on vertex vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:03,839 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 22:28:03,839 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:03,839 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:03,839 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:03,840 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9D2E7DE72BB445DFBF4E678D79FE076A, vertexId=vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 22:28:03,840 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9D2E7DE72BB445DFBF4E678D79FE076A, vertexId=vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:03,842 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9D2E7DE72BB445DFBF4E678D79FE076A, vertexName=F46378B714124D539310573022FADA58, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 22:28:03,842 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9D2E7DE72BB445DFBF4E678D79FE076A, vertexId=vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:03,844 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:03,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F46378B714124D539310573022FADA58, vertexId=vertex_1463524083659_0001_1_01, initRequestedTime=1463524083832, initedTime=1463524083844, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 22:28:03,844 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58] transitioned from NEW to INITED due to event V_INIT
2016-05-17 22:28:03,844 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:03,844 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B8DE3565C5434AD2A9D556FB4027A0CB=forAllWorkUnits=true, update=[1]}
2016-05-17 22:28:03,844 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] parallelism set to 1
2016-05-17 22:28:03,845 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 22:28:03,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524083659_0001_1_00, eventCount=1
2016-05-17 22:28:03,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 22:28:03,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, vertexId=vertex_1463524083659_0001_1_00, initRequestedTime=1463524083831, initedTime=1463524083845, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 22:28:03,846 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] with distanceFromRoot: 0
2016-05-17 22:28:03,846 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 22:28:03,847 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:03,847 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:03,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524083659_0001_1_00, startRequestedTime=1463524083847, startedTime=1463524083847
2016-05-17 22:28:03,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:03,848 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:03,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:03,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:TASK_STARTED]: vertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, taskId=task_1463524083659_0001_1_00_000000, scheduledTime=1463524083848, launchTime=1463524083848
2016-05-17 22:28:03,849 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] submitted tez dag to app master: application_1463524083659_0001, with dag id: dag_1463524083659_0001_1
2016-05-17 22:28:03,849 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524083659_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:03,850 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463524083659_0001_1_00 for vertex: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58] numStartedSources: 1 numSources: 1
2016-05-17 22:28:03,850 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58] with distanceFromRoot: 1
2016-05-17 22:28:03,850 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:03,852 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ECBD9F5C6AB4452CAC3DE7F303DB08FF in vertex: F46378B714124D539310573022FADA58
2016-05-17 22:28:03,852 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F46378B714124D539310573022FADA58 with 1 source tasks and 1 pending tasks
2016-05-17 22:28:03,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463524083659_0001_1_01, startRequestedTime=1463524083850, startedTime=1463524083850
2016-05-17 22:28:03,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 22:28:03,853 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58] transitioned from INITED to RUNNING due to event V_START
2016-05-17 22:28:03,854 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20] test/(1/1) ...AssertionsPass/value/pass, VertexName: ECBD9F5C6AB4452CAC3DE7F303DB08FF, VertexParallelism: 1, TaskAttemptID:attempt_1463524083659_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B8DE3565C5434AD2A9D556FB4027A0CB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F46378B714124D539310573022FADA58, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:03,856 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524083659_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:03,856 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:03,857 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 22:28:03,858 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 22:28:03,858 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 22:28:03,858 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524083659_0001_00_000001
2016-05-17 22:28:03,858 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:03,860 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524083659_0001_00_000001
2016-05-17 22:28:03,860 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:03,861 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:03,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:03,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524083659_0001_00_000001, launchTime=1463524083861
2016-05-17 22:28:03,862 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524083659_0001_00_000001
2016-05-17 22:28:03,862 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524083659_0001_1_00_000000_0] to container: [container_1463524083659_0001_00_000001]
2016-05-17 22:28:03,862 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:03,862 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524083659_0001_00_000001 given task: attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:03,863 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524083659_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:03,863 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524083659_0001_1_00_000000_0] started. Is using containerId: [container_1463524083659_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 22:28:03,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, taskAttemptId=attempt_1463524083659_0001_1_00_000000_0, startTime=1463524083863, containerId=container_1463524083659_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524083659_0001_00_000001/teamcity, completedLogs=
2016-05-17 22:28:03,863 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524083659_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:03,863 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:03,863 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524083659_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:03 Starting to run new task attempt: attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:03,864 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:03,864 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:03,864 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:03,864 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:03,864 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:03,864 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:03,865 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:03,865 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20] test/(1/1) ...AssertionsPass/value/pass, VertexName: ECBD9F5C6AB4452CAC3DE7F303DB08FF, VertexParallelism: 1, TaskAttemptID:attempt_1463524083659_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B8DE3565C5434AD2A9D556FB4027A0CB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F46378B714124D539310573022FADA58, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 22:28:03,865 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=718798848
2016-05-17 22:28:03,868 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524083659_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 22:28:03,869 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:03,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:03,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B8DE3565C5434AD2A9D556FB4027A0CB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 22:28:03,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:03,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B8DE3565C5434AD2A9D556FB4027A0CB
2016-05-17 22:28:03,872 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B8DE3565C5434AD2A9D556FB4027A0CB
2016-05-17 22:28:03,873 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:03,874 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 22:28:03,875 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B8DE3565C5434AD2A9D556FB4027A0CB
2016-05-17 22:28:03,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B8DE3565C5434AD2A9D556FB4027A0CB
2016-05-17 22:28:03,876 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F46378B714124D539310573022FADA58, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 22:28:03,876 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:03,876 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F46378B714124D539310573022FADA58
2016-05-17 22:28:03,879 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ECBD9F5C6AB4452CAC3DE7F303DB08FF, ordinal: 0
2016-05-17 22:28:03,879 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 22:28:03,879 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ECBD9F5C6AB4452CAC3DE7F303DB08FF, mem on start (mb), free: 474, total: 685, max: 685
2016-05-17 22:28:03,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:03,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:03,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:03,879 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F46378B714124D539310573022FADA58
2016-05-17 22:28:03,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F46378B714124D539310573022FADA58
2016-05-17 22:28:03,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:03,880 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:03,881 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 22:28:03,881 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:03,881 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 22:28:03,881 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:03,881 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 718798848, TotalAvailable: 481595228, TotalRequested/TotalJVMHeap:0.15
2016-05-17 22:28:03,881 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B8DE3565C5434AD2A9D556FB4027A0CB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 22:28:03,882 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F46378B714124D539310573022FADA58, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 22:28:03,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:03,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B8DE3565C5434AD2A9D556FB4027A0CB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:03,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:03,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:03,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B8DE3565C5434AD2A9D556FB4027A0CB
2016-05-17 22:28:03,883 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:03,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B8DE3565C5434AD2A9D556FB4027A0CB
2016-05-17 22:28:03,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:03,884 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:03,884 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 22:28:03,884 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2956acd0
2016-05-17 22:28:03,885 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2956acd0
2016-05-17 22:28:03,885 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 22:28:03,886 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 22:28:03,887 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:03,887 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: B8DE3565C5434AD2A9D556FB4027A0CB
2016-05-17 22:28:03,888 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:'method']], id: E53193340A234F94A76D2B96CCE7679C
2016-05-17 22:28:03,888 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:'method']] E53193340A234F94A76D2B96CCE7679C
2016-05-17 22:28:03,889 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@7480f19e; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3d76f7d3; comparator=cascading.tuple.hadoop.util.TupleComparator@4d885d2b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 22:28:03,889 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 22:28:03,890 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 22:28:03,890 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:04,004 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 22:28:04,004 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 22:28:04,017 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 22:28:04,017 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 22:28:04,017 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 22:28:04,018 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 22:28:04,018 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 22:28:04,020 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-05-17 22:28:04,020 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 22:28:04,020 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ECBD9F5C6AB4452CAC3DE7F303DB08FF, all 1 inputs ready in: 00:00:00.000
2016-05-17 22:28:04,024 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:04,024 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ECBD9F5C6AB4452CAC3DE7F303DB08FF, mem on close (mb), free: 610, total: 739, max: 739
2016-05-17 22:28:04,024 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 22:28:04,024 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 22:28:04,025 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 693; bufvoid = 104857600
2016-05-17 22:28:04,025 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214372(104857488); length = 25/6553600
2016-05-17 22:28:04,028 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 22:28:04 Completed running task attempt: attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:04,032 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524083659_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:04,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463524083659_0001_1_00, eventCount=1
2016-05-17 22:28:04,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=7, GC_TIME_MILLIS=114, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463524084021, Process_Duration=3, Process_End_Time=1463524084024, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-05-17 22:28:04,034 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:04,034 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:04,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:04,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, taskAttemptId=attempt_1463524083659_0001_1_00_000000_0, startTime=1463524083863, finishTime=1463524084034, timeTaken=171, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=114, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524084021, Process_Duration=3, Process_End_Time=1463524084024, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:04,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524083659_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:04,035 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524083659_0001_00_000001
2016-05-17 22:28:04,035 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:04,035 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524083659_0001_1_00_000000_0
2016-05-17 22:28:04,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:TASK_FINISHED]: vertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, taskId=task_1463524083659_0001_1_00_000000, startTime=1463524083863, finishTime=1463524084036, timeTaken=173, status=SUCCEEDED, successfulAttemptID=attempt_1463524083659_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=114, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524084021, Process_Duration=3, Process_End_Time=1463524084024, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-05-17 22:28:04,037 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524083659_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:04,037 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] : 1
2016-05-17 22:28:04,037 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:04,037 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:04,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:04,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:VERTEX_FINISHED]: vertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, vertexId=vertex_1463524083659_0001_1_00, initRequestedTime=1463524083831, initedTime=1463524083845, startRequestedTime=1463524083847, startedTime=1463524083847, finishTime=1463524084037, timeTaken=190, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=685, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=7, GC_TIME_MILLIS=114, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=7, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524084021, Process_Duration=3, Process_End_Time=1463524084024, Read_Duration=1, Tuples_Read=10, Tuples_Written=7, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1463524083863, firstTasksToStart=[ task_1463524083659_0001_1_00_000000 ], lastTaskFinishTime=1463524084034, lastTasksToFinish=[ task_1463524083659_0001_1_00_000000 ], minTaskDuration=171, maxTaskDuration=171, avgTaskDuration=171.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524083659_0001_1_00_000000 ], longestDurationTasks=[ task_1463524083659_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:04,039 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:04,039 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:04,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524083659_0001_00_000001, stoppedTime=1463524084039, exitStatus=0
2016-05-17 22:28:04,040 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:04,040 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58] attempt: attempt_1463524083659_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 22:28:04,040 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F46378B714124D539310573022FADA58
2016-05-17 22:28:04,040 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:04,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:04,040 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:04,040 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463524083659_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 22:28:04,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:TASK_STARTED]: vertexName=F46378B714124D539310573022FADA58, taskId=task_1463524083659_0001_1_01_000000, scheduledTime=1463524084041, launchTime=1463524084041
2016-05-17 22:28:04,041 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524083659_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 22:28:04,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 22:28:04,041 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:04,042 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 22:28:04,042 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20] test/(1/1) ...AssertionsPass/value/pass, VertexName: F46378B714124D539310573022FADA58, VertexParallelism: 1, TaskAttemptID:attempt_1463524083659_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9D2E7DE72BB445DFBF4E678D79FE076A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:04,042 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524083659_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 22:28:04,042 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 22:28:04,043 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463524083659_0001_00_000002
2016-05-17 22:28:04,043 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 22:28:04,044 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463524083659_0001_00_000002
2016-05-17 22:28:04,044 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(145)) - DAGAppMaster metrics system already initialized!
2016-05-17 22:28:04,045 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 22:28:04,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 22:28:04,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463524083659_0001_00_000002, launchTime=1463524084045
2016-05-17 22:28:04,046 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524083659_0001_00_000002
2016-05-17 22:28:04,046 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524083659_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 22:28:04,046 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524083659_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:28:04,047 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524083659_0001_00_000001, Exiting
2016-05-17 22:28:04,046 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463524083659_0001_1_01_000000_0] to container: [container_1463524083659_0001_00_000002]
2016-05-17 22:28:04,047 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524083659_0001_00_000001
2016-05-17 22:28:04,047 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524083659_0001_00_000001
2016-05-17 22:28:04,047 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 22:28:04,047 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524083659_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:28:04,048 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463524083659_0001_1_01_000000_0] started. Is using containerId: [container_1463524083659_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 22:28:04,048 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463524083659_0001_00_000002 given task: attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 22:28:04,048 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524083659_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F46378B714124D539310573022FADA58, taskAttemptId=attempt_1463524083659_0001_1_01_000000_0, startTime=1463524084048, containerId=container_1463524083659_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463524083659_0001_00_000002/teamcity, completedLogs=
2016-05-17 22:28:04,049 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524083659_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 22:28:04,049 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524083659_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 22:28:04 Starting to run new task attempt: attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,049 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,050 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:04,050 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 22:28:04,050 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 22:28:04,050 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 22:28:04,050 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 22:28:04,050 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 22:28:04,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20] test/(1/1) ...AssertionsPass/value/pass, VertexName: F46378B714124D539310573022FADA58, VertexParallelism: 1, TaskAttemptID:attempt_1463524083659_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9D2E7DE72BB445DFBF4E678D79FE076A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 22:28:04,050 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=774897664
2016-05-17 22:28:04,049 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524083659_0001_00_000001 completed successfully
2016-05-17 22:28:04,051 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 22:28:04,051 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 22:28:04,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:04,053 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463524083659_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 22:28:04,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:04,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ECBD9F5C6AB4452CAC3DE7F303DB08FF, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 22:28:04,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 22:28:04,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:04,057 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 697407872, based on INPUT_BUFFER_factor: 0.9
2016-05-17 22:28:04,057 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 22:28:04,057 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 697407872, type: INPUT, componentVertexName: ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:04,058 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 22:28:04,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:04,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9D2E7DE72BB445DFBF4E678D79FE076A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 22:28:04,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 22:28:04,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9D2E7DE72BB445DFBF4E678D79FE076A
2016-05-17 22:28:04,066 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 22:28:04,066 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9D2E7DE72BB445DFBF4E678D79FE076A
2016-05-17 22:28:04,067 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F46378B714124D539310573022FADA58, ordinal: 1
2016-05-17 22:28:04,069 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F46378B714124D539310573022FADA58, mem on start (mb), free: 605, total: 739, max: 739
2016-05-17 22:28:04,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 22:28:04,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 22:28:04,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 22:28:04,076 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 22:28:04,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9D2E7DE72BB445DFBF4E678D79FE076A
2016-05-17 22:28:04,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 22:28:04,077 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 22:28:04,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 22:28:04,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 22:28:04,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 22:28:04,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 22:28:04,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 697407872, TotalRequestedScaled: 6.437611126153847E8, TotalJVMHeap: 774897664, TotalAvailable: 519181434, TotalRequested/TotalJVMHeap:0.90
2016-05-17 22:28:04,077 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ECBD9F5C6AB4452CAC3DE7F303DB08FF, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=697407872, allocated=519181434
2016-05-17 22:28:04,078 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9D2E7DE72BB445DFBF4E678D79FE076A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 22:28:04,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 22:28:04,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ECBD9F5C6AB4452CAC3DE7F303DB08FF being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 22:28:04,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:04,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 22:28:04,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 22:28:04,079 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 22:28:04,079 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 22:28:04,080 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ECBD9F5C6AB4452CAC3DE7F303DB08FF with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 22:28:04,080 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=697407872, postMergeMem=0, RuntimeTotalAvailable=519181434. Updated to: ShuffleMem=519181434, postMergeMem: 0
2016-05-17 22:28:04,080 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=519181434, maxSingleShuffleLimit=129795360, mergeThreshold=467263296, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 22:28:04,081 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 22:28:04,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:04,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 22:28:04,085 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,085 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463524083659_0001_1_00_000000_0_10081, runDuration: 154001, ]
2016-05-17 22:28:04,086 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463524083659_0001_1_00_000000_0_10081] to DISK_DIRECT, CompressedSize=653, DecompressedSize=649,EndTime=1463524084086, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 22:28:04,087 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:04,087 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 22:28:04,087 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [ECBD9F5C6AB4452CAC3DE7F303DB08FF] #14 in 1ms
2016-05-17 22:28:04,087 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ECBD9F5C6AB4452CAC3DE7F303DB08FF] #14, status:false, isInterrupted:false
2016-05-17 22:28:04,089 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:'method']]
2016-05-17 22:28:04,090 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:'method']] streamed: true, id: E53193340A234F94A76D2B96CCE7679C
2016-05-17 22:28:04,090 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsPass/value/pass"], id: 9D2E7DE72BB445DFBF4E678D79FE076A
2016-05-17 22:28:04,090 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 22:28:04,090 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsPass/value/pass"]
2016-05-17 22:28:04,091 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:'method']] E53193340A234F94A76D2B96CCE7679C, for 1 inputs
2016-05-17 22:28:04,091 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 653 bytes from disk
2016-05-17 22:28:04,091 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:04,091 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 22:28:04,091 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 633 bytes
2016-05-17 22:28:04,091 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:04,091 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 22:28:04,092 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F46378B714124D539310573022FADA58, all 1 inputs ready in: 00:00:00.001
2016-05-17 22:28:04,095 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4d73b5c0; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@67a177f9
2016-05-17 22:28:04,096 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 22:28:04,096 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 22:28:04,096 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,097 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463524083659_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 22:28:04,097 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,097 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F46378B714124D539310573022FADA58, mem on close (mb), free: 602, total: 739, max: 739
2016-05-17 22:28:04,097 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ECBD9F5C6AB4452CAC3DE7F303DB08FF
2016-05-17 22:28:04,098 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 22:28:04,098 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 22:28:04,098 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 22:28:04,098 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 22:28:04,099 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 22:28:04 Completed running task attempt: attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,099 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463524083659_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 22:28:04,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463524084092, Process_Duration=5, Process_End_Time=1463524084097, Read_Duration=0, Tuples_Read=7, Tuples_Written=1, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=1]]
2016-05-17 22:28:04,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F46378B714124D539310573022FADA58, taskAttemptId=attempt_1463524083659_0001_1_01_000000_0, startTime=1463524084048, finishTime=1463524084100, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524084092, Process_Duration=5, Process_End_Time=1463524084097, Read_Duration=0, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:28:04,101 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 22:28:04,101 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463524083659_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 22:28:04,101 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 22:28:04,101 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:04,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 22:28:04,101 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463524083659_0001_00_000002
2016-05-17 22:28:04,101 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463524083659_0001_1_01_000000_0
2016-05-17 22:28:04,102 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 22:28:04,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:TASK_FINISHED]: vertexName=F46378B714124D539310573022FADA58, taskId=task_1463524083659_0001_1_01_000000, startTime=1463524084048, finishTime=1463524084102, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463524083659_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524084092, Process_Duration=5, Process_End_Time=1463524084097, Read_Duration=0, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1
2016-05-17 22:28:04,103 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463524083659_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 22:28:04,103 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58] : 1
2016-05-17 22:28:04,103 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:04,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 22:28:04,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:VERTEX_FINISHED]: vertexName=F46378B714124D539310573022FADA58, vertexId=vertex_1463524083659_0001_1_01, initRequestedTime=1463524083832, initedTime=1463524083844, startRequestedTime=1463524083850, startedTime=1463524083850, finishTime=1463524084103, timeTaken=253, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=685, FILE_BYTES_WRITTEN=19, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=7, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, OUTPUT_RECORDS=1, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463524084092, Process_Duration=5, Process_End_Time=1463524084097, Read_Duration=0, Tuples_Read=7, Tuples_Written=1, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=1, vertexStats=firstTaskStartTime=1463524084048, firstTasksToStart=[ task_1463524083659_0001_1_01_000000 ], lastTaskFinishTime=1463524084100, lastTasksToFinish=[ task_1463524083659_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463524083659_0001_1_01_000000 ], longestDurationTasks=[ task_1463524083659_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 22:28:04,104 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 22:28:04,104 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 22:28:04,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463524083659_0001_00_000002, stoppedTime=1463524084104, exitStatus=0
2016-05-17 22:28:04,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 22:28:04,105 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 22:28:04,105 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 22:28:04,105 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463524083659_0001_1
2016-05-17 22:28:04,105 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463524083659_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 22:28:04,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463524083659_0001_1
2016-05-17 22:28:04,106 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463524083659_0001_1_00 [ECBD9F5C6AB4452CAC3DE7F303DB08FF]
2016-05-17 22:28:04,106 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9D2E7DE72BB445DFBF4E678D79FE076A for vertex: vertex_1463524083659_0001_1_01 [F46378B714124D539310573022FADA58]
2016-05-17 22:28:04,105 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463524083659_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 22:28:04,108 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463524083659_0001_00_000002, Exiting
2016-05-17 22:28:04,108 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463524083659_0001_00_000002
2016-05-17 22:28:04,108 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463524083659_0001_00_000002
2016-05-17 22:28:04,108 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463524083659_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 22:28:04,109 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463524083659_0001_00_000002 completed successfully
2016-05-17 22:28:04,116 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463524083659_0001_1, queueSize=0
2016-05-17 22:28:04,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 22:28:04,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463524083659_0001_1][Event:DAG_FINISHED]: dagId=dag_1463524083659_0001_1, startTime=1463524083830, finishTime=1463524084114, timeTaken=284, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2891, FILE_BYTES_WRITTEN=704, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=7, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=14, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=114, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1549795328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=8, OUTPUT_BYTES=693, OUTPUT_BYTES_WITH_OVERHEAD=649, OUTPUT_BYTES_PHYSICAL=653, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=653, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=653, SHUFFLE_BYTES_DECOMPRESSED=649, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=653, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927048168113, Process_Duration=8, Process_End_Time=2927048168121, Read_Duration=1, Tuples_Read=17, Tuples_Written=8, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=1
2016-05-17 22:28:04,116 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463524083659_0001_1 finished with state: SUCCEEDED
2016-05-17 22:28:04,116 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463524083659_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 22:28:04,117 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463524083659_0001_00_000002
2016-05-17 22:28:04,117 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463524083659_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 22:28:04 Completed Dag: dag_1463524083659_0001_1
2016-05-17 22:28:04,118 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463524083659_0001_1, dagState=SUCCEEDED
2016-05-17 22:28:04,118 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463524083659_0001_1_post
2016-05-17 22:28:04,119 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 22:28:04,119 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 22:28:04,123 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...AssertionsPass/value/pass, applicationId=application_1463524083659_0001
2016-05-17 22:28:04,125 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 22:28:04,125 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 22:28:04,125 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 22:28:04,125 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 22:28:04,125 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 22:28:04,125 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 22:28:04,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 22:28:04,126 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 22:28:04,126 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 22:28:04,126 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 22:28:04,127 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 22:28:04,127 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 22:28:04,127 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 22:28:04,131 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/tmp/.staging/56C82BA2E3FF4D10B1974E3E48F6DF81/E344DF637C4344D8A2D4692DA4597A20/.tez/application_1463524083659_0001
2016-05-17 22:28:04,132 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output/hadoop2-tez/appliedassertionsplatform/testValueAssertionsPass/value/pass/_temporary
2016-05-17 22:28:04,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [test] completed in: 00:00.495
2016-05-17 22:28:04,166 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.5.x-tez-0.6.x/build/test/output//hadoop2-tez/appliedassertionsplatform/testValueAssertionsPass/value/pass